root
/
DataX
mirror of https://github.com/alibaba/DataX.git
1
0
Fork 0
Code Issues Actions Packages Projects Releases Wiki Activity

fix:oceanbase datasource support special characters

This commit is contained in:
xxsc0529 2025-05-27 17:39:39 +08:00
parent 1bc342e6b4
commit c1e34c96c4
4 changed files with 16 additions and 15 deletions
Show Stats Download Patch File Download Diff File Expand all files Collapse all files

2
README.md
View File

@ -41,7 +41,7 @@ DataX目前已经有了比较全面的插件体系，主流的RDBMS数据库、N
|--------------|---------------------------|:---------:|:---------:|:----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------:|
| RDBMS 关系型数据库 | MySQL | √ | √ | [读](https://github.com/alibaba/DataX/blob/master/mysqlreader/doc/mysqlreader.md) 、[写](https://github.com/alibaba/DataX/blob/master/mysqlwriter/doc/mysqlwriter.md) |
| | Oracle | √ | √ | [读](https://github.com/alibaba/DataX/blob/master/oraclereader/doc/oraclereader.md) 、[写](https://github.com/alibaba/DataX/blob/master/oraclewriter/doc/oraclewriter.md) |
| | OceanBase | √ | √ | [读](https://open.oceanbase.com/docs/community/oceanbase-database/V3.1.0/use-datax-to-full-migration-data-to-oceanbase) 、[写](https://open.oceanbase.com/docs/community/oceanbase-database/V3.1.0/use-datax-to-full-migration-data-to-oceanbase) |
| | OceanBase | √ | √ | [读](https://github.com/alibaba/DataX/blob/master/oceanbasev10reader/doc/oceanbasev10reader.md) 、[写](https://github.com/alibaba/DataX/blob/master/oceanbasev10writer/doc/oceanbasev10writer.md) |
| | SQLServer | √ | √ | [读](https://github.com/alibaba/DataX/blob/master/sqlserverreader/doc/sqlserverreader.md) 、[写](https://github.com/alibaba/DataX/blob/master/sqlserverwriter/doc/sqlserverwriter.md) |
| | PostgreSQL | √ | √ | [读](https://github.com/alibaba/DataX/blob/master/postgresqlreader/doc/postgresqlreader.md) 、[写](https://github.com/alibaba/DataX/blob/master/postgresqlwriter/doc/postgresqlwriter.md) |
| | DRDS | √ | √ | [读](https://github.com/alibaba/DataX/blob/master/drdsreader/doc/drdsreader.md) 、[写](https://github.com/alibaba/DataX/blob/master/drdswriter/doc/drdswriter.md) |

15
oceanbasev10reader/doc/oceanbasev10reader.md
View File

@ -228,4 +228,17 @@ OceanbaseV10Reader插件实现了从Oceanbase V1.0读取数据。在底层实现
| 4 | 55862 | 17.60 |
| 5 | 70860 | 22.31 |
#
## 5常见问题
###
4.1 oracle模式下报错Invalid fatch size
```
Caused by: java.sql.SQLSyntaxErrorException: (conn=2498) invalid fetch size. in Oracle mode, extendOracleResultSetClass is ineffective if useOraclePrepareExecute is set to true or usePieceData is set to true
at com.oceanbase.jdbc.internal.util.exceptions.ExceptionFactory.createException(ExceptionFactory.java:110)
at com.oceanbase.jdbc.internal.util.exceptions.ExceptionFactory.create(ExceptionFactory.java:211)
at com.oceanbase.jdbc.OceanBaseStatement.setFetchSize(OceanBaseStatement.java:1599)
at com.alibaba.datax.plugin.reader.oceanbasev10reader.ext.ReaderTask.doRead(ReaderTask.java:270)
... 5 more
```
该错误常发生更换了高版本的oceanbase-client.jar驱动，高版本的驱动未来提高效率，增加了oracle预处理语句行为。这个机制和setFetchSize冲突。
#### 解决方案
在jdbcUrl中配置extendOracleResultSetClass=true可解决这个冲突。

2
oceanbasev10reader/src/main/java/com/alibaba/datax/plugin/reader/oceanbasev10reader/ext/Constant.java
View File

@ -8,6 +8,4 @@ public class Constant {
public static String WEAK_READ_QUERY_SQL_TEMPLATE_WITHOUT_WHERE = "select /*+read_consistency(weak)*/ %s from %s ";
public static String WEAK_READ_QUERY_SQL_TEMPLATE = "select /*+read_consistency(weak)*/ %s from %s where (%s)";
public static final int ORACLE_DEFAULT_FETCH_SIZE = 1024;
}

10
oceanbasev10reader/src/main/java/com/alibaba/datax/plugin/reader/oceanbasev10reader/ext/ReaderTask.java
View File

@ -30,9 +30,6 @@ import org.slf4j.LoggerFactory;
import java.sql.*;
import java.util.ArrayList;
import java.util.List;
import java.util.Objects;
import static com.alibaba.datax.plugin.reader.oceanbasev10reader.ext.Constant.ORACLE_DEFAULT_FETCH_SIZE;
public class ReaderTask extends CommonRdbmsReader.Task {
private static final Logger LOG = LoggerFactory.getLogger(ReaderTask.class);
@ -259,14 +256,7 @@ public class ReaderTask extends CommonRdbmsReader.Task {
}
}
// 打开流式接口
//oceanbase的oracle模式FetchSize同oracle插件的设置，否则会报错。invalid fetch size. in Oracle mode,
// 具体请看com.oceanbase.jdbc.OceanBaseStatement#setFetchSize
if (context.getFetchSize()==Integer.MIN_VALUE
&& Objects.equals(compatibleMode, ObReaderUtils.OB_COMPATIBLE_MODE_ORACLE)){
ps.setFetchSize(ORACLE_DEFAULT_FETCH_SIZE);
}else {
ps.setFetchSize(context.getFetchSize());
}
rs = ps.executeQuery();
ResultSetMetaData metaData = rs.getMetaData();
int columnNumber = metaData.getColumnCount();