对从 Oracle 到 PostgreSQL 的部分数据库迁移的对象进行多级依赖性分析 - AWS Prescriptive Guidance

本文属于机器翻译版本。若本译文内容与英语原文存在差异,则一律以英文原文为准。

对从 Oracle 到 PostgreSQL 的部分数据库迁移的对象进行多级依赖性分析

anuradha chintha,Amazon Web Services

摘要

这种模式描述了在将部分 Oracle 数据库迁移到亚马逊关系数据库服务 (Amazon RDS) 或亚马逊 Aurora PostgreSQL 时,系统地识别和管理系统依赖关系的重要性。在部分迁移中,只有一部分数据库对象和原始数据库中的数据会被迁移,而源数据库将继续运行并为依赖未迁移组件的应用程序提供服务

在处理将应用程序与上游和下游依赖关系紧密耦合的大型数据库时,必须确定和分析迁移的范围。要开始部分迁移,请确定作用域对象,包括表、触发器、视图、存储过程、函数和包。范围识别过程遵循全面的方法:

  • 第一级作用域对象是通过应用程序代码中的直接引用和特定于模块的关键作业来识别的。

  • 二级对象是通过全面的依赖关系分析得出的。

了解系统不同部分的交互方式后,就可以更好地规划正确的数据库组件移动顺序,降低迁移失败的风险。下表列出了不同类型的依赖关系分析。

分析类型

重点领域

用途

对象依赖关系

  • 视图

  • 存储过程

  • 函数

  • 触发

识别数据库对象与其分层结构之间的关系

区段依赖关系

  • 外键关系

  • 主钥匙链

  • 跨架构引用

映射数据关系并保持引用完整性

安全依赖关系

  • 用户权限

  • 角色层次结构

  • 对象权限

确保适当的访问控制迁移和安全维护

访问模式

  • 读取操作

  • 写入操作

确定数据库交互模式

要保持源系统和目标系统之间的一致性,请在过渡期间建立数据同步机制。您还必须修改应用程序代码和函数,以处理源 Oracle 数据库和目标 PostgreSQL 数据库之间的数据分发。

先决条件和限制

先决条件

  • 活跃的 AWS 账户

  • 甲骨文数据库(来源)

  • 亚马逊 RDS 或亚马逊 Aurora PostgreSQL(目标)

产品版本

  • 甲骨文 19c 或更高版本

  • PostgreSQL 16 或更高版本

架构

源技术堆栈

  • 甲骨文 19c 或更高版本

目标技术堆栈

  • 亚马逊 RDS 或亚马逊 Aurora PostgreSQL

目标架构

下图显示了从本地 Oracle 数据库迁移到 Amazon RDS for Oracle 的迁移过程,其中涉及以下内容:

  • 识别数据库依赖关系

  • 使用 AWS Schema Conversion Tool (AWS SCT) 迁移数据库代码和对象

  • 使用 AWS Database Migration Service (AWS DMS) 迁移数据

  • 使用变更数据捕获 (CDC) 复制正在进行的更改 AWS DMS

有关更多信息,请参阅 AWS 文档 AWS Schema Conversion Tool中的AWS Database Migration Service 与集成

工具

AWS 服务

  • 适用于 Oracle 的 Amazon Relational Database Service(Amazon RDS)可帮助您在中设置、操作和扩展 Oracle 关系数据库 AWS Cloud。

  • Amazon Aurora 是与 MySQL 和 PostgreSQL 兼容的完全托管式的云端关系数据库引擎。

  • AWS Schema Conversion Tool (AWS SCT) 通过自动将源数据库架构和大部分自定义代码转换为与目标数据库兼容的格式来支持异构数据库迁移。

  • AWS Database Migration Service (AWS DMS) 可帮助您将数据存储迁移到云和本地设置的组合中, AWS Cloud 或者迁移到云端和本地设置的组合之间。

其他服务

  • Oracle SQL Developer 是一个集成的开发环境,可简化传统部署和基于云的部署中 Oracle 数据库的开发和管理。对于这种模式,你可以使用 SQL*Plus

最佳实践

有关预置和迁移 Oracle 数据库的最佳实践,请参阅迁移到 Amazon RDS for Oracle 的最佳实践。

操作说明

Task描述所需技能

创建对象表。

确定对应用程序功能至关重要的对象,然后创建一个名为的表DEPENDENT_ANALYSIS_BASELINE。将每个对象的记录添加到表中。有关示例,请参阅其他信息部分。

数据工程师、数据库管理员

创建数据库过程。

使用DBA_DEPENDENCIES表中的数据创建一个名sp_object_dependency_analysis为的存储过程,用于分析双向(向前和向后)的对象依赖关系。有关示例,请参阅其他信息部分。

数据工程师、数据库管理员

运行该程序。

在每个连续级别上运行脚本,直到找不到新的对象依赖关系。所有依赖关系和级别都存储在DEPENDENT_ANALYSIS_BASELINE表中。

数据库管理员、数据工程师
Task描述所需技能

创建依赖关系表。

创建名为的区段级依赖关系表。REFERENTIAL_ANALYSIS_BASELINE发现所有对象级依赖关系后,DEPENDENT_ANALYSIS_BASELINE通过查询表来检查其父表。DBA_CONSTRAINT

排除其他表引用基线表的依赖关系。回填处理这些关系。以下是一个脚本示例:

CREATE TABLE REFERENTIAL_ANALYSIS_BASELINE (CHILD_OWNER VARCHAR2(50 BYTE), CHILD_NAME VARCHAR2(100 BYTE), PARENT_OWNER VARCHAR2(50 BYTE), PARENT_NAME VARCHAR2(50 BYTE), REFERENCE_PATH VARCHAR2(1000 BYTE));
数据工程师、数据库管理员

创建数据库过程。

创建一个名为的过程SP_OBJECT_REFERENTIAL_ANALYSIS,然后为所有已识别的对象生成参考分析。有关示例,请参阅其他信息部分。

数据工程师、数据库管理员

运行该程序。

运行该过程以获取参照依赖关系。在中生成参考分析对象的详细信息。REFERENTIAL_ANALYSIS_BASELINE

数据工程师、数据库管理员
Task描述所需技能

创建用于读取和写入对象的表。

使用以下脚本创建名为的读取对象表TABLE_READ_OBJECT_DETAILS和名为的写入对象表TABLE_WRITE_OBJECT_DETAILS

CREATE TABLE TABLE_READ_OBJECT_DETAILS (OWNER VARCHAR2(50 BYTE), TAB_NAME VARCHAR2(50 BYTE), READER_OWNER VARCHAR2(50 BYTE), READER_NAME VARCHAR2(50 BYTE), READER_TYPE VARCHAR2(50 BYTE));
CREATE TABLE TABLE_WRITE_OBJECT_DETAILS (TABLE_NAME VARCHAR2(100 BYTE), WRITEOBJ_OWNER VARCHAR2(100 BYTE), WRITEOBJ_NAME VARCHAR2(100 BYTE), WRITEOBJ_TYPE VARCHAR2(100 BYTE), LINE VARCHAR2(100 BYTE), TEXT VARCHAR2(4000 BYTE), OWNER VARCHAR2(50 BYTE));
数据工程师、数据库管理员

创建分析程序。

分别创建SP_WRITER_OBJECTS_ANALYSIS用于分析读取对象和写入对象的过程SP_READER_OBJECTS_ANALYSIS和。这些过程使用模式匹配来查找相关对象。 有关示例,请参阅 “其他信息” 部分。

数据工程师、数据库管理员

运行程序。

运行以下过程以识别依赖对象。

数据库管理员、数据工程师
Task描述所需技能

创建用于查看权限的表。

创建用于分析名为的权限的表OBJECT_PRIVS_ANALYSIS。要以递归方式捕获DEPENDENT_ANALYSIS_BASELINE表中的对象权限,请使用以下脚本:

CREATE TABLE OBJECT_PRIVS_ANALYSIS (OWNER VARCHAR2(50 BYTE), OBJECT_NAME VARCHAR2(50 BYTE), USER_NAME VARCHAR2(50 BYTE), PRIVS VARCHAR2(50 BYTE));
数据工程师、数据库管理员

创建查看权限的程序。

创建名为的过程SP_OBJECT_PRIVS_ANALYSIS。为已识别的对象生成权限分析。有关示例,请参阅其他信息部分。

数据库管理员、数据工程师

运行该程序。

运行该过程以将它们捕获到OBJECT_PRIVS_ANALYSIS表中。

数据库管理员、数据工程师

故障排除

事务解决方案

无法访问字典表

确保创建分析对象的用户可以访问 DBA 表。

相关资源

AWS 文档

其他文档

其他信息

的脚本 DEPENDENT_ANALYSIS_BASELINE

CREATE TABLE DEPENDENT_ANALYSIS_BASELINE (OWNER VARCHAR2(128 BYTE) NOT NULL ENABLE, OBJECT_NAME VARCHAR2(128 BYTE) NOT NULL ENABLE, OBJECT_TYPE VARCHAR2(20 BYTE), DEPEDNCY_LEVEL NUMBER, PROJECT_NEED VARCHAR2(20 BYTE), CATAGORY VARCHAR2(4000 BYTE), COMMENTS VARCHAR2(4000 BYTE), CATAGORY1 CLOB, COMMENTS1 CLOB, CUSTOMER_COMMENTS VARCHAR2(1000 BYTE), BACKFILL_TO_GUS VARCHAR2(1000 BYTE), BACKFILL_NEAR_REAL_TIME_OR_BATCH VARCHAR2(1000 BYTE), PK_EXISTS VARCHAR2(3 BYTE), UI_EXISTS VARCHAR2(3 BYTE), LOB_EXISTS VARCHAR2(3 BYTE), MASTER_LINK VARCHAR2(100 BYTE), CONSTRAINT PK_DEPENDENT_ANALYSIS_BASELINE PRIMARY KEY (OWNER,OBJECT_NAME,OBJECT_TYPE));

的程序 SP_WRITER_OBJECTS_ANALYSIS

CREATE OR REPLACE PROCEDURE SP_WRITER_OBJECTS_ANALYSIS IS BEGIN EXECUTE IMMEDIATE 'TRUNCATE TABLE TABLE_WRITE_OBJECT_DETAILS'; FOR I IN (SELECT OWNER, OBJECT_NAME FROM DEPENDENT_ANALYSIS_BASELINE WHERE OBJECT_TYPE = 'TABLE') LOOP INSERT INTO TABLE_WRITE_OBJECT_DETAILS(OWNER, TABLE_NAME, WRITEOBJ_OWNER, WRITEOBJ_NAME, WRITEOBJ_TYPE, LINE, TEXT) SELECT DISTINCT I.OWNER, I.OBJECT_NAME, OWNER WRITEOBJ_OWNER, NAME, TYPE, LINE, TRIM(TEXT) FROM DBA_SOURCE WHERE UPPER(TEXT) LIKE '%' || I.OBJECT_NAME || '%' AND (UPPER(TEXT) LIKE '%INSERT%' || I.OBJECT_NAME || '%' OR UPPER(TEXT) LIKE '%UPDATE%' || I.OBJECT_NAME || '%' OR UPPER(TEXT) LIKE '%DELETE%' || I.OBJECT_NAME || '%' OR UPPER(TEXT) LIKE '%UPSERT%' || I.OBJECT_NAME || '%' OR UPPER(TEXT) LIKE '%MERGE%' || I.OBJECT_NAME || '%') AND UPPER(TEXT) NOT LIKE '%PROCEDURE%' AND UPPER(TEXT) NOT LIKE 'PROCEDURE%' AND UPPER(TEXT) NOT LIKE '%FUNCTION%' AND UPPER(TEXT) NOT LIKE 'FUNCTION%' AND UPPER(TEXT) NOT LIKE '%TRIGGER%' AND UPPER(TEXT) NOT LIKE 'TRIGGER%' AND UPPER(TRIM(TEXT)) NOT LIKE '%AFTER UPDATE%' AND UPPER(TRIM(TEXT)) NOT LIKE 'BEFORE UPDATE%' AND UPPER(TRIM(TEXT)) NOT LIKE 'BEFORE INSERT%' AND UPPER(TRIM(TEXT)) NOT LIKE 'AFTER INSERT%' AND UPPER(TRIM(TEXT)) NOT LIKE 'BEFORE DELETE%' AND UPPER(TRIM(TEXT)) NOT LIKE 'AFTER DELETE%' AND UPPER(TRIM(TEXT)) NOT LIKE '%GGLOGADM.GG_LOG_ERROR%' AND (TRIM(TEXT) NOT LIKE '/*%' AND TRIM(TEXT) NOT LIKE '--%' ) AND (OWNER, NAME, TYPE) IN (SELECT OWNER, NAME, TYPE FROM DBA_DEPENDENCIES WHERE REFERENCED_NAME = I.OBJECT_NAME); END LOOP; END;

的脚本 SP_READER_OBJECTS_ANALYSIS

CREATE OR REPLACE PROCEDURE SP_READER_OBJECTS_ANALYSIS IS BEGIN EXECUTE IMMEDIATE 'TRUNCATE TABLE TABLE_READ_OBJECT_DETAILS'; FOR I IN (SELECT OWNER, OBJECT_NAME FROM DEPENDENT_ANALYSIS_BASELINE WHERE OBJECT_TYPE = 'TABLE') LOOP INSERT INTO TABLE_READ_OBJECT_DETAILS SELECT DISTINCT i.owner, i.object_name, owner, name, type FROM dba_dependencies WHERE referenced_name = I.OBJECT_NAME AND referenced_type = 'TABLE' AND type NOT IN ('SYNONYM', 'MATERIALIZED VIEW', 'VIEW') AND (owner, name, type) NOT IN ( SELECT DISTINCT owner, trigger_name, 'TRIGGER' FROM dba_triggers WHERE table_name = I.OBJECT_NAME AND table_owner = i.owner UNION ALL SELECT DISTINCT owner, name, type FROM dba_source WHERE upper(text) LIKE '%' || I.OBJECT_NAME || '%' AND (upper(text) LIKE '%INSERT %' || I.OBJECT_NAME || '%' OR upper(text) LIKE '%UPDATE% ' || I.OBJECT_NAME || '%' OR upper(text) LIKE '%DELETE %' || I.OBJECT_NAME || '%' OR upper(text) LIKE '%UPSERT %' || I.OBJECT_NAME || '%' OR upper(text) LIKE '%MERGE %' || I.OBJECT_NAME || '%') AND upper(text) NOT LIKE '%PROCEDURE %' AND upper(text) NOT LIKE 'PROCEDURE %' AND upper(text) NOT LIKE '%FUNCTION %' AND upper(text) NOT LIKE 'FUNCTION %' AND upper(text) NOT LIKE '%TRIGGER %' AND upper(text) NOT LIKE 'TRIGGER %' AND upper(trim(text)) NOT LIKE 'BEFORE INSERT %' AND upper(trim(text)) NOT LIKE 'BEFORE UPDATE %' AND upper(trim(text)) NOT LIKE 'BEFORE DELETE %' AND upper(trim(text)) NOT LIKE 'AFTER INSERT %' AND upper(trim(text)) NOT LIKE 'AFTER UPDATE %' AND upper(trim(text)) NOT LIKE 'AFTER DELETE %' AND (trim(text) NOT LIKE '/*%' AND trim(text) NOT LIKE '--%')); END LOOP; END;

的脚本 SP_OBJECT_REFERENTIAL_ANALYSIS

CREATE OR REPLACE PROCEDURE SP_OBJECT_REFERENTIAL_ANALYSIS IS BEGIN EXECUTE IMMEDIATE 'TRUNCATE TABLE REFERENTIAL_ANALYSIS_BASELINE'; INSERT INTO REFERENTIAL_ANALYSIS_BASELINE WITH rel AS ( SELECT DISTINCT c.owner, c.table_name, c.r_owner r_owner, (SELECT table_name FROM dba_constraints WHERE constraint_name = c.r_constraint_name AND owner = c.r_owner) r_table_name FROM dba_constraints c WHERE constraint_type = 'R' AND c.owner NOT IN (SELECT username FROM dba_users WHERE oracle_maintained = 'Y') AND c.r_owner NOT IN (SELECT username FROM dba_users WHERE oracle_maintained = 'Y')), tab_list AS ( SELECT OWNER, object_name FROM DEPENDENT_ANALYSIS_BASELINE WHERE UPPER(OBJECT_TYPE) = 'TABLE') SELECT DISTINCT owner child_owner, table_name child, r_owner parent_owner, r_table_name parent, SYS_CONNECT_BY_PATH(r_table_name, ' -> ') || ' -> ' || table_name PATH FROM rel START WITH (r_owner, r_table_name) IN (SELECT * FROM tab_list) CONNECT BY NOCYCLE (r_owner, r_table_name) = ((PRIOR owner, PRIOR table_name)) UNION SELECT DISTINCT owner child_owner, table_name child, r_owner parent_owner, r_table_name parent, SYS_CONNECT_BY_PATH(table_name, ' -> ') || ' -> ' || r_table_name PATH FROM rel START WITH (owner, table_name) IN (SELECT * FROM tab_list) CONNECT BY NOCYCLE (owner, table_name) = ((PRIOR r_owner, PRIOR r_table_name)); END;

的脚本 SP_OBJECT_PRIVS_ANALYSIS

CREATE OR REPLACE PROCEDURE SP_OBJECT_PRIVS_ANALYSIS IS V_SQL VARCHAR2(4000); V_CNT NUMBER; BEGIN V_SQL := 'TRUNCATE TABLE OBJECT_PRIVS_ANALYSIS'; EXECUTE IMMEDIATE V_SQL; FOR I IN (SELECT OWNER, OBJECT_NAME FROM DEPENDENT_ANALYSIS_BASELINE WHERE OBJECT_TYPE = 'TABLE') LOOP INSERT INTO OBJECT_PRIVS_ANALYSIS(OWNER, OBJECT_NAME, USER_NAME, PRIVS) WITH obj_to_role AS ( SELECT DISTINCT GRANTEE role_name, DECODE(privilege, 'SELECT', 'READ', 'REFERENCE', 'READ', 'INSERT', 'WRITE', 'UPDATE', 'WRITE', 'DELETE', 'WRITE', privilege) privs FROM DBA_TAB_PRIVS t, DBA_ROLES r WHERE OWNER = I.OWNER AND TYPE = 'TABLE' AND TABLE_NAME = I.OBJECT_NAME AND t.GRANTEE = r.ROLE AND r.ROLE IN (SELECT ROLE FROM DBA_ROLES WHERE ORACLE_MAINTAINED = 'N') ) SELECT I.OWNER, I.OBJECT_NAME, grantee, privs FROM ( -- Recursively Role to User mapping with privilege SELECT DISTINCT grantee, privs FROM (SELECT rp.granted_role, rp.grantee, privs, (SELECT DECODE(COUNT(*), 0, 'ROLE', 'USER') FROM (SELECT 'User' FROM DBA_users WHERE username = rp.GRANTEE)) grantee_type FROM DBA_role_privs rp, obj_to_role r WHERE rp.granted_role = r.role_name AND grantee IN ((SELECT USERNAME FROM DBA_USERS WHERE ORACLE_MAINTAINED = 'N') UNION (SELECT ROLE FROM DBA_ROLES WHERE ORACLE_MAINTAINED = 'N')) AND granted_role IN (SELECT ROLE FROM DBA_ROLES WHERE ORACLE_MAINTAINED = 'N') START WITH granted_role IN (SELECT DISTINCT role_name FROM obj_to_role) CONNECT BY granted_role = PRIOR grantee) WHERE grantee_type = 'USER' ) UNION ( -- Direct Object grants to User SELECT I.OWNER, I.OBJECT_NAME, GRANTEE, DECODE(privilege, 'SELECT', 'READ', 'REFERENCE', 'READ', 'INSERT', 'WRITE', 'UPDATE', 'WRITE', 'DELETE', 'WRITE', privilege) privs FROM DBA_TAB_PRIVS, DBA_USERS WHERE GRANTEE = USERNAME AND OWNER = I.OWNER AND TYPE = 'TABLE' AND TABLE_NAME = I.OBJECT_NAME ) ORDER BY 2 DESC; END LOOP; END;

的程序 SP_OBJECT_DEPENDENCY_ANALYSIS

CREATE OR REPLACE PROCEDURE SP_OBJECT_DEPENDENCY_ANALYSIS (v_level NUMBER) IS TYPE typ IS RECORD ( schema VARCHAR2(100), obj_type VARCHAR2(100), obj_name VARCHAR2(100), path VARCHAR2(5000) ); TYPE array IS TABLE OF typ; l_data array; c SYS_REFCURSOR; l_errors NUMBER; l_errno NUMBER; l_msg VARCHAR2(4000); l_idx NUMBER; l_level NUMBER; BEGIN l_level := v_level + 1; OPEN c FOR WITH obj_list AS ( SELECT owner schema_name, object_type, object_name FROM DEPENDENT_ANALYSIS_BASELINE WHERE depedncy_level = v_level ), fw_dep_objects AS ( SELECT level lvl, owner, name, type, referenced_owner, referenced_name, referenced_type, SYS_CONNECT_BY_PATH(name, ' -> ') || ' -> ' || referenced_name PATH FROM dba_dependencies START WITH (owner, CASE WHEN type = 'PACKAGE BODY' THEN 'PACKAGE' ELSE type END, name) IN (SELECT schema_name, object_type, object_name FROM obj_list) CONNECT BY NOCYCLE (owner, type, name) = ((PRIOR referenced_owner, PRIOR referenced_type, PRIOR referenced_name)) ), bw_dep_objects AS ( SELECT level lvl, owner, name, type, referenced_owner, referenced_name, referenced_type, SYS_CONNECT_BY_PATH(name, ' <- ') || ' <- ' || referenced_name PATH FROM dba_dependencies START WITH (referenced_owner, CASE WHEN referenced_type = 'PACKAGE BODY' THEN 'PACKAGE' ELSE referenced_type END, referenced_name) IN (SELECT schema_name, object_type, object_name FROM obj_list) CONNECT BY NOCYCLE (referenced_owner, referenced_type, referenced_name) = ((PRIOR owner, PRIOR type, PRIOR name)) ) SELECT * FROM ( (SELECT DISTINCT referenced_owner schema, referenced_type obj_type, referenced_name obj_name, path FROM fw_dep_objects) UNION (SELECT DISTINCT owner schema, type obj_type, name obj_name, path FROM bw_dep_objects) ) WHERE schema IN (SELECT username FROM all_users WHERE oracle_maintained = 'N') ORDER BY obj_type; LOOP FETCH c BULK COLLECT INTO l_data LIMIT 100; BEGIN FORALL i IN 1..l_data.count SAVE EXCEPTIONS INSERT INTO DEPENDENT_ANALYSIS_BASELINE ( owner, object_name, object_type, catagory, depedncy_level, project_need, comments ) VALUES ( l_data(i).schema, l_data(i).obj_name, CASE WHEN l_data(i).obj_type = 'PACKAGE BODY' THEN 'PACKAGE' ELSE l_data(i).obj_type END, 'level ' || l_level || ' dependency', l_level, '', 'from dependency proc' || l_data(i).path ); EXCEPTION WHEN OTHERS THEN l_errors := sql%bulk_exceptions.count; FOR i IN 1..l_errors LOOP l_errno := sql%bulk_exceptions(i).error_code; l_msg := SQLERRM(-l_errno); l_idx := sql%bulk_exceptions(i).error_index; UPDATE DEPENDENT_ANALYSIS_BASELINE SET catagory1 = catagory1 || ', found in level' || l_level || ' dependent of ' || l_data(l_idx).path, comments1 = comments1 || ', from dependency proc exception ' || l_data(i).path WHERE owner = l_data(l_idx).schema AND object_name = l_data(l_idx).obj_name AND object_type = l_data(l_idx).obj_type; END LOOP; END; EXIT WHEN c%NOTFOUND; END LOOP; CLOSE c; END;