doris 和StarRocks 导入导出数据配置

原创已于 2025-06-26 11:28:50 修改 · 887 阅读

12 ·

CC 4.0 BY-SA版权

文章标签：

#database #hadoop #big data

于 2025-06-25 20:32:55 首次发布

一、StarRocks 导数据到hdfs

EXPORT TABLE database.table
TO “hdfs://namenode/tmp/demo/table”
WITH BROKER
(
“username”=“username”,
“password”=“password”
);

二、StarRocks 导数据到oss

EXPORT TABLE database.table
TO “oss://broke/aa/”
WITH BROKER
(
“fs.oss.accessKeyId” = “accessKeyId”,
“fs.oss.accessKeySecret” = “accessKeySecret”,
“fs.oss.endpoint” = “endpoint”
);

三、看导出状态
SHOW EXPORT FROM databaseORDER BY StartTime DESC LIMIT 30

四、从oss导入数据到sr
LOAD LABEL database.tableName
(
DATA INFILE(“oss://brack/tableName/*”)
INTO TABLE tableName
FORMAT AS “csv”
(table fields，逗号隔开)
)
WITH BROKER
(
“fs.oss.accessKeyId” = “accessKeyId”,
“fs.oss.accessKeySecret” = “accessKeySecret”,
“fs.oss.endpoint” = “endpoint”
)
PROPERTIES
(
“timeout” = “3600”
);

五、doris 导出到hdfs
EXPORT TABLE database.tableName
TO ‘hdfs://namenode/tmp/demo/’
PROPERTIES (
‘label’ = ‘exportTool_tableName’
,‘column_separator’ = ‘|\x01|’
,‘line_delimiter’ = ‘|\x03|’
,‘exec_mem_limit’ = ‘2147483648’
,‘format’ = ‘CSV’
,‘timeout’ = ‘18000’ – 单位秒
)
WITH BROKER ‘broker_cluster’ (
‘max_filter_ratio’ = ‘0’
,‘username’ = ‘sauser’
,‘password’ = ‘’
,‘hadoop.username’ = ‘sauser’
,‘dfs.nameservices’ = ‘bihdp’
,‘dfs.namenode.rpc-address.bihdp.nn_1’ = ‘IP:8020’
,‘dfs.namenode.rpc-address.bihdp.nn_2’ = ‘IP:8020’
,‘dfs.client.failover.proxy.provider’ = ‘org.apache.hadoop.hdfs.server.namenode.ha.ConfiguredFailoverProxyProvider’
,‘dfs.ha.namenodes.bihdp’ = ‘nn_1,nn_2’
)