Kafka to ADB Connector installation
To install Kafka to ADB Connector, follow the steps:
-
Make sure that the Kafka to ADB service is installed. For information on how to add services to ADB clusters, see Add services.
-
Connect to the database via psql (or any other client program):
$ psql adb -
Check that the new extension
kadb_fdwis available — via the psql meta-command\dx:\dxThe output contains the
kadb_fdwextension:List of installed extensions Name | Version | Schema | Description -----------------------------+---------+------------+--------------------------------------------------------------------- adb_fdw | 1.0 | public | foreign-data wrapper for remote ADB servers arenadata_toolkit | 1.1 | public | extension is used for manipulation of objects created by adb-bundle dblink | 1.1 | public | connect to other PostgreSQL databases from within a database diskquota | 2.2 | public | Disk Quota Main Program gp_parallel_retrieve_cursor | 1.0 | pg_catalog | Retrieve results of cursor in parallel gptkh | 0.14 | public | gptkh kadb_fdw | 0.16 | kadb | Kafka-ADB foreign data wrapper plpgsql | 1.0 | pg_catalog | PL/pgSQL procedural language postgis | 2.5.4 | public | PostGIS geometry, geography, and raster spatial types and functions pxf | 2.0 | public | Extension which allows to access unmanaged data pxf_fdw | 1.0 | public | PXF Foreign Data Wrapper for Greenplum tkh_fdw | 1.0 | public | ClickHouse Foreign Data Wrapper for Greenplum (12 rows) -
You can also check that the new foreign data wrapper
kadb_fdwis available using the following SQL query against thepg_catalog.pg_foreign_data_wrappersystem table:SELECT * FROM pg_catalog.pg_foreign_data_wrapper;The result:
fdwname | fdwowner | fdwhandler | fdwvalidator | fdwacl | fdwoptions ---------------+----------+------------+--------------+--------+-------------------------------------------------------------------------------------------- dblink_fdw | 10 | 0 | 16508 | | jdbc_pxf_fdw | 10 | 20336 | 20337 | | {protocol=jdbc,"mpp_execute=all segments"} hdfs_pxf_fdw | 10 | 20336 | 20337 | | {protocol=hdfs,"mpp_execute=all segments"} hive_pxf_fdw | 10 | 20336 | 20337 | | {protocol=hive,"mpp_execute=all segments"} hbase_pxf_fdw | 10 | 20336 | 20337 | | {protocol=hbase,"mpp_execute=all segments"} s3_pxf_fdw | 10 | 20336 | 20337 | | {protocol=s3,"mpp_execute=all segments"} gs_pxf_fdw | 10 | 20336 | 20337 | | {protocol=gs,"mpp_execute=all segments"} adl_pxf_fdw | 10 | 20336 | 20337 | | {protocol=adl,"mpp_execute=all segments"} wasbs_pxf_fdw | 10 | 20336 | 20337 | | {protocol=wasbs,"mpp_execute=all segments"} file_pxf_fdw | 10 | 20336 | 20337 | | {protocol=file,"mpp_execute=all segments"} tkh_fdw | 10 | 37116 | 37115 | | {protocol=tkh_clickhouse,"mpp_execute=all segments",format=text} adb_fdw | 10 | 37136 | 37137 | | kadb_fdw | 10 | 28724 | 28726 | | {"mpp_execute=all segments",k_initial_offset=0,k_automatic_offsets=true,k_latency_ms=2000} (13 rows) -
Ensure that the kadb schema is available — via the psql meta-command
\dnS+:\dnS+The result:
Name | Owner | Access privileges | Description --------------------+---------+--------------------+------------------------------------------------------------- arenadata_toolkit | gpadmin | gpadmin=UC/gpadmin+| | | =U/gpadmin | diskquota | gpadmin | | gp_toolkit | gpadmin | gpadmin=UC/gpadmin+| | | =U/gpadmin | information_schema | gpadmin | gpadmin=UC/gpadmin+| | | =U/gpadmin | kadb | gpadmin | | madlib | gpadmin | | pg_aoseg | gpadmin | | Reserved schema for Append Only segment list and eof tables pg_bitmapindex | gpadmin | | Reserved schema for internal relations of bitmap indexes pg_catalog | gpadmin | gpadmin=UC/gpadmin+| system catalog schema | | =U/gpadmin | pg_toast | gpadmin | | reserved schema for TOAST tables public | gpadmin | gpadmin=UC/gpadmin+| standard public schema | | =UC/gpadmin | (11 rows)