Ankit Malik
308 lines
24 KiB
Plaintext
308 lines
24 KiB
Plaintext
2026-06-22 10:05:56 | INFO | ================================================================================
|
|
2026-06-22 10:05:56 | INFO | Hello from data-move Python data pipeline!
|
|
2026-06-22 10:05:56 | INFO | Pipeline Run Date: 2026-06-21
|
|
2026-06-22 10:05:56 | INFO | Connecting to databases...
|
|
2026-06-22 10:05:57 | INFO | <sqlalchemy.engine.cursor.CursorResult object at 0x000001F93F950730>
|
|
2026-06-22 10:06:00 | INFO | <sqlalchemy.engine.cursor.CursorResult object at 0x000001F9409FC7D0>
|
|
2026-06-22 10:06:03 | INFO | Database connections established
|
|
2026-06-22 10:06:03 | INFO | Collecting MIDs for: 2026-06-21
|
|
2026-06-22 10:06:05 | INFO | Found 569 MIDs
|
|
2026-06-22 10:06:07 | INFO | ================================================================================
|
|
2026-06-22 10:06:07 | INFO | Processing Table: Master_VisibilityReason | Table type is -: DIMENSION | Based on -none and operation is used -DELETE+INSERT
|
|
2026-06-22 10:06:07 | INFO | Fetching Data from sql server for table-: Master_VisibilityReason ..............
|
|
2026-06-22 10:06:07 | INFO | Fetching data from sql server for Master table......
|
|
2026-06-22 10:06:07 | INFO | Fetching in progress ....
|
|
2026-06-22 10:06:31 | ERROR | Failed processing table Master_VisibilityReason
|
|
Traceback (most recent call last):
|
|
File "C:\Users\ankitm\AppData\Local\uv\cache\environments-v2\main2-f1515603462f26e4\Lib\site-packages\sqlalchemy\engine\base.py", line 1969, in _exec_single_context
|
|
self.dialect.do_execute(
|
|
~~~~~~~~~~~~~~~~~~~~~~~^
|
|
cursor, str_statement, effective_parameters, context
|
|
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
|
|
)
|
|
^
|
|
File "C:\Users\ankitm\AppData\Local\uv\cache\environments-v2\main2-f1515603462f26e4\Lib\site-packages\sqlalchemy\engine\default.py", line 952, in do_execute
|
|
cursor.execute(statement, parameters)
|
|
~~~~~~~~~~~~~~^^^^^^^^^^^^^^^^^^^^^^^
|
|
pyodbc.OperationalError: ('08S01', '[08S01] [Microsoft][ODBC Driver 18 for SQL Server]TCP Provider: A connection attempt failed because the connected party did not properly respond after a period of time, or established connection failed because connected host has failed to respond.\r\n (10060) (SQLExecDirectW); [08S01] [Microsoft][ODBC Driver 18 for SQL Server]Communication link failure (10060)')
|
|
|
|
The above exception was the direct cause of the following exception:
|
|
|
|
Traceback (most recent call last):
|
|
File "D:\data_move\main2.py", line 161, in main
|
|
df = fetch_data(sql_engine ,table_name,table_type)
|
|
File "D:\data_move\src\dim.py", line 45, in fetch_data
|
|
df = pl.read_database(
|
|
query=sql,
|
|
connection=engine
|
|
)
|
|
File "C:\Users\ankitm\AppData\Local\uv\cache\environments-v2\main2-f1515603462f26e4\Lib\site-packages\polars\io\database\functions.py", line 284, in read_database
|
|
return cx.execute(
|
|
~~~~~~~~~~^
|
|
query=query,
|
|
^^^^^^^^^^^^
|
|
options=execute_options,
|
|
^^^^^^^^^^^^^^^^^^^^^^^^
|
|
).to_polars(
|
|
^
|
|
File "C:\Users\ankitm\AppData\Local\uv\cache\environments-v2\main2-f1515603462f26e4\Lib\site-packages\polars\io\database\_executor.py", line 546, in execute
|
|
result = cursor_execute(query, **options)
|
|
File "C:\Users\ankitm\AppData\Local\uv\cache\environments-v2\main2-f1515603462f26e4\Lib\site-packages\sqlalchemy\engine\base.py", line 1421, in execute
|
|
return meth(
|
|
self,
|
|
distilled_parameters,
|
|
execution_options or NO_OPTIONS,
|
|
)
|
|
File "C:\Users\ankitm\AppData\Local\uv\cache\environments-v2\main2-f1515603462f26e4\Lib\site-packages\sqlalchemy\sql\elements.py", line 526, in _execute_on_connection
|
|
return connection._execute_clauseelement(
|
|
~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~^
|
|
self, distilled_params, execution_options
|
|
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
|
|
)
|
|
^
|
|
File "C:\Users\ankitm\AppData\Local\uv\cache\environments-v2\main2-f1515603462f26e4\Lib\site-packages\sqlalchemy\engine\base.py", line 1643, in _execute_clauseelement
|
|
ret = self._execute_context(
|
|
dialect,
|
|
...<8 lines>...
|
|
cache_hit=cache_hit,
|
|
)
|
|
File "C:\Users\ankitm\AppData\Local\uv\cache\environments-v2\main2-f1515603462f26e4\Lib\site-packages\sqlalchemy\engine\base.py", line 1848, in _execute_context
|
|
return self._exec_single_context(
|
|
~~~~~~~~~~~~~~~~~~~~~~~~~^
|
|
dialect, context, statement, parameters
|
|
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
|
|
)
|
|
^
|
|
File "C:\Users\ankitm\AppData\Local\uv\cache\environments-v2\main2-f1515603462f26e4\Lib\site-packages\sqlalchemy\engine\base.py", line 1988, in _exec_single_context
|
|
self._handle_dbapi_exception(
|
|
~~~~~~~~~~~~~~~~~~~~~~~~~~~~^
|
|
e, str_statement, effective_parameters, cursor, context
|
|
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
|
|
)
|
|
^
|
|
File "C:\Users\ankitm\AppData\Local\uv\cache\environments-v2\main2-f1515603462f26e4\Lib\site-packages\sqlalchemy\engine\base.py", line 2365, in _handle_dbapi_exception
|
|
raise sqlalchemy_exception.with_traceback(exc_info[2]) from e
|
|
File "C:\Users\ankitm\AppData\Local\uv\cache\environments-v2\main2-f1515603462f26e4\Lib\site-packages\sqlalchemy\engine\base.py", line 1969, in _exec_single_context
|
|
self.dialect.do_execute(
|
|
~~~~~~~~~~~~~~~~~~~~~~~^
|
|
cursor, str_statement, effective_parameters, context
|
|
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
|
|
)
|
|
^
|
|
File "C:\Users\ankitm\AppData\Local\uv\cache\environments-v2\main2-f1515603462f26e4\Lib\site-packages\sqlalchemy\engine\default.py", line 952, in do_execute
|
|
cursor.execute(statement, parameters)
|
|
~~~~~~~~~~~~~~^^^^^^^^^^^^^^^^^^^^^^^
|
|
sqlalchemy.exc.OperationalError: (pyodbc.OperationalError) ('08S01', '[08S01] [Microsoft][ODBC Driver 18 for SQL Server]TCP Provider: A connection attempt failed because the connected party did not properly respond after a period of time, or established connection failed because connected host has failed to respond.\r\n (10060) (SQLExecDirectW); [08S01] [Microsoft][ODBC Driver 18 for SQL Server]Communication link failure (10060)')
|
|
[SQL: with Master_VisibilityReason (ProjectId,MenuId,ReasonId,Reason,CreateDate,Createby)
|
|
as (
|
|
|
|
select DISTINCT '40148' as ProjectId,MenuId,VisibilityReasonId,VisibilityReason,getdate(),'SP-Pius'
|
|
FROM OneApp_KelloggsMT.dbo.Master_VisibilityReason
|
|
)
|
|
select * from Master_VisibilityReason]
|
|
(Background on this error at: https://sqlalche.me/e/20/e3q8)
|
|
2026-06-22 10:10:52 | INFO | ================================================================================
|
|
2026-06-22 10:10:52 | INFO | Hello from data-move Python data pipeline!
|
|
2026-06-22 10:10:52 | INFO | Pipeline Run Date: 2026-06-21
|
|
2026-06-22 10:10:53 | INFO | Connecting to databases...
|
|
2026-06-22 10:10:53 | INFO | <sqlalchemy.engine.cursor.CursorResult object at 0x000001500A41C730>
|
|
2026-06-22 10:10:55 | INFO | <sqlalchemy.engine.cursor.CursorResult object at 0x000001500B56C7D0>
|
|
2026-06-22 10:10:56 | INFO | Database connections established
|
|
2026-06-22 10:10:56 | INFO | Collecting MIDs for: 2026-06-21
|
|
2026-06-22 10:10:56 | INFO | Found 569 MIDs
|
|
2026-06-22 10:10:57 | INFO | ================================================================================
|
|
2026-06-22 10:10:57 | INFO | Processing Table: Master_VisibilityReason | Table type is -: DIMENSION | Based on -none and operation is used -DELETE+INSERT
|
|
2026-06-22 10:10:57 | INFO | Fetching Data from sql server for table-: Master_VisibilityReason ..............
|
|
2026-06-22 10:10:57 | INFO | Fetching data from sql server for Master table......
|
|
2026-06-22 10:10:57 | INFO | Fetching in progress ....
|
|
2026-06-22 10:10:57 | INFO | Fetched 17 rows from SQL Server
|
|
2026-06-22 10:10:57 | INFO | Fetched total row -: 17 from sql server for table-:Master_VisibilityReason ...........!!!
|
|
2026-06-22 10:10:57 | INFO | Fetched 17 rows
|
|
2026-06-22 10:10:57 | INFO | Creating table Master_VisibilityReason
|
|
2026-06-22 10:10:57 | INFO | Table ready: Master_VisibilityReason
|
|
2026-06-22 10:10:57 | INFO | _ _ _ _Inserting data into clickhouse db from sql server_ _ _ _
|
|
2026-06-22 10:10:58 | INFO | Master_VisibilityReason: inserted 17 rows into ClickHouse
|
|
2026-06-22 10:10:58 | INFO | Master_VisibilityReason loaded successfully (17 rows)
|
|
2026-06-22 10:10:58 | INFO | ================================================================================
|
|
2026-06-22 10:10:58 | INFO | Pipeline Completed Successfully
|
|
2026-06-22 10:10:58 | INFO | ================================================================================
|
|
2026-06-22 10:11:04 | INFO | ================================================================================
|
|
2026-06-22 10:11:04 | INFO | Hello from data-move Python data pipeline!
|
|
2026-06-22 10:11:04 | INFO | Pipeline Run Date: 2026-06-21
|
|
2026-06-22 10:11:04 | INFO | Connecting to databases...
|
|
2026-06-22 10:11:05 | INFO | <sqlalchemy.engine.cursor.CursorResult object at 0x000002DFCC410730>
|
|
2026-06-22 10:11:06 | INFO | <sqlalchemy.engine.cursor.CursorResult object at 0x000002DFCD4CC7D0>
|
|
2026-06-22 10:11:07 | INFO | Database connections established
|
|
2026-06-22 10:11:07 | INFO | Collecting MIDs for: 2026-06-21
|
|
2026-06-22 10:11:07 | INFO | Found 569 MIDs
|
|
2026-06-22 10:11:08 | INFO | ================================================================================
|
|
2026-06-22 10:11:08 | INFO | Processing Table: Master_VisibilityReason | Table type is -: DIMENSION | Based on -none and operation is used -DELETE+INSERT
|
|
2026-06-22 10:11:08 | INFO | Fetching Data from sql server for table-: Master_VisibilityReason ..............
|
|
2026-06-22 10:11:08 | INFO | Fetching data from sql server for Master table......
|
|
2026-06-22 10:11:08 | INFO | Fetching in progress ....
|
|
2026-06-22 10:11:08 | INFO | Fetched 17 rows from SQL Server
|
|
2026-06-22 10:11:08 | INFO | Fetched total row -: 17 from sql server for table-:Master_VisibilityReason ...........!!!
|
|
2026-06-22 10:11:08 | INFO | Fetched 17 rows
|
|
2026-06-22 10:11:08 | INFO | Truncated table Master_VisibilityReason
|
|
2026-06-22 10:11:08 | INFO | _ _ _ _Inserting data into clickhouse db from sql server_ _ _ _
|
|
2026-06-22 10:11:08 | INFO | Master_VisibilityReason: inserted 17 rows into ClickHouse
|
|
2026-06-22 10:11:08 | INFO | Master_VisibilityReason loaded successfully (17 rows)
|
|
2026-06-22 10:11:08 | INFO | ================================================================================
|
|
2026-06-22 10:11:08 | INFO | Pipeline Completed Successfully
|
|
2026-06-22 10:11:08 | INFO | ================================================================================
|
|
2026-06-22 11:07:54 | INFO | ================================================================================
|
|
2026-06-22 11:07:54 | INFO | Hello from data-move Python data pipeline!
|
|
2026-06-22 11:07:54 | INFO | Pipeline Run Date: 2026-06-21
|
|
2026-06-22 11:07:54 | INFO | Connecting to databases...
|
|
2026-06-22 11:07:56 | INFO | <sqlalchemy.engine.cursor.CursorResult object at 0x000001689BE04730>
|
|
2026-06-22 11:08:00 | INFO | <sqlalchemy.engine.cursor.CursorResult object at 0x000001689CEBC7D0>
|
|
2026-06-22 11:08:01 | INFO | Database connections established
|
|
2026-06-22 11:08:01 | INFO | Collecting MIDs for: 2026-06-21
|
|
2026-06-22 11:08:02 | INFO | Found 569 MIDs
|
|
2026-06-22 11:08:42 | INFO | ================================================================================
|
|
2026-06-22 11:08:42 | INFO | Hello from data-move Python data pipeline!
|
|
2026-06-22 11:08:42 | INFO | Pipeline Run Date: 2026-06-21
|
|
2026-06-22 11:08:42 | INFO | Connecting to databases...
|
|
2026-06-22 11:08:44 | INFO | <sqlalchemy.engine.cursor.CursorResult object at 0x0000015B77AE4730>
|
|
2026-06-22 11:08:46 | INFO | <sqlalchemy.engine.cursor.CursorResult object at 0x0000015B78B8C7D0>
|
|
2026-06-22 11:08:47 | INFO | Database connections established
|
|
2026-06-22 11:08:47 | INFO | Collecting MIDs for: 2026-06-21
|
|
2026-06-22 11:08:48 | INFO | Found 569 MIDs
|
|
2026-06-22 11:08:48 | INFO | ================================================================================
|
|
2026-06-22 11:08:48 | INFO | Processing Table: Master_VisibilityReason | Table type is -: DIMENSION | Based on -none and operation is used -DELETE+INSERT
|
|
2026-06-22 11:08:48 | INFO | Fetching Data from sql server for table-: Master_VisibilityReason ..............
|
|
2026-06-22 11:08:48 | INFO | Fetching data from sql server for Master table......
|
|
2026-06-22 11:08:48 | INFO | Fetching in progress ....
|
|
2026-06-22 11:08:48 | INFO | Fetched 17 rows from SQL Server
|
|
2026-06-22 11:08:48 | INFO | Fetched total row -: 17 from sql server for table-:Master_VisibilityReason ...........!!!
|
|
2026-06-22 11:08:48 | INFO | Fetched 17 rows
|
|
2026-06-22 11:08:48 | INFO | Truncated table Master_VisibilityReason
|
|
2026-06-22 11:08:48 | INFO | _ _ _ _Inserting data into clickhouse db from sql server_ _ _ _
|
|
2026-06-22 11:08:49 | INFO | Master_VisibilityReason: inserted 17 rows into ClickHouse
|
|
2026-06-22 11:08:49 | INFO | Master_VisibilityReason loaded successfully (17 rows)
|
|
2026-06-22 11:08:49 | INFO | ================================================================================
|
|
2026-06-22 11:08:49 | INFO | Processing Table: Master_VisibilityDefinition | Table type is -: DIMENSION | Based on -none and operation is used -DELETE+INSERT
|
|
2026-06-22 11:08:49 | INFO | Fetching Data from sql server for table-: Master_VisibilityDefinition ..............
|
|
2026-06-22 11:08:49 | INFO | Fetching data from sql server for Master table......
|
|
2026-06-22 11:08:49 | INFO | Fetching in progress ....
|
|
2026-06-22 11:08:49 | INFO | Fetched 861 rows from SQL Server
|
|
2026-06-22 11:08:49 | INFO | Fetched total row -: 861 from sql server for table-:Master_VisibilityDefinition ...........!!!
|
|
2026-06-22 11:08:49 | INFO | Fetched 861 rows
|
|
2026-06-22 11:08:49 | INFO | Creating table Master_VisibilityDefinition
|
|
2026-06-22 11:08:49 | INFO | Table ready: Master_VisibilityDefinition
|
|
2026-06-22 11:08:49 | INFO | _ _ _ _Inserting data into clickhouse db from sql server_ _ _ _
|
|
2026-06-22 11:08:49 | INFO | Master_VisibilityDefinition: inserted 861 rows into ClickHouse
|
|
2026-06-22 11:08:49 | INFO | Master_VisibilityDefinition loaded successfully (861 rows)
|
|
2026-06-22 11:08:49 | INFO | ================================================================================
|
|
2026-06-22 11:08:49 | INFO | Processing Table: Master_Salesterritorylayer | Table type is -: DIMENSION | Based on -none and operation is used -DELETE+INSERT
|
|
2026-06-22 11:08:49 | INFO | Fetching Data from sql server for table-: Master_Salesterritorylayer ..............
|
|
2026-06-22 11:08:49 | INFO | Fetching data from sql server for Master table......
|
|
2026-06-22 11:08:49 | INFO | Fetching in progress ....
|
|
2026-06-22 11:08:49 | INFO | Fetched 33 rows from SQL Server
|
|
2026-06-22 11:08:49 | INFO | Fetched total row -: 33 from sql server for table-:Master_Salesterritorylayer ...........!!!
|
|
2026-06-22 11:08:49 | INFO | Fetched 33 rows
|
|
2026-06-22 11:08:50 | INFO | Creating table Master_Salesterritorylayer
|
|
2026-06-22 11:08:50 | INFO | Table ready: Master_Salesterritorylayer
|
|
2026-06-22 11:08:50 | INFO | _ _ _ _Inserting data into clickhouse db from sql server_ _ _ _
|
|
2026-06-22 11:08:50 | INFO | Master_Salesterritorylayer: inserted 33 rows into ClickHouse
|
|
2026-06-22 11:08:50 | INFO | Master_Salesterritorylayer loaded successfully (33 rows)
|
|
2026-06-22 11:08:50 | INFO | ================================================================================
|
|
2026-06-22 11:08:50 | INFO | Pipeline Completed Successfully
|
|
2026-06-22 11:08:50 | INFO | ================================================================================
|
|
2026-06-22 11:08:56 | INFO | ================================================================================
|
|
2026-06-22 11:08:56 | INFO | Hello from data-move Python data pipeline!
|
|
2026-06-22 11:08:56 | INFO | Pipeline Run Date: 2026-06-21
|
|
2026-06-22 11:08:56 | INFO | Connecting to databases...
|
|
2026-06-22 11:08:57 | INFO | <sqlalchemy.engine.cursor.CursorResult object at 0x00000214E3374730>
|
|
2026-06-22 11:08:59 | INFO | <sqlalchemy.engine.cursor.CursorResult object at 0x00000214E44CC7D0>
|
|
2026-06-22 11:09:00 | INFO | Database connections established
|
|
2026-06-22 11:09:00 | INFO | Collecting MIDs for: 2026-06-21
|
|
2026-06-22 11:09:00 | INFO | Found 569 MIDs
|
|
2026-06-22 11:09:00 | INFO | ================================================================================
|
|
2026-06-22 11:09:00 | INFO | Processing Table: Master_VisibilityReason | Table type is -: DIMENSION | Based on -none and operation is used -DELETE+INSERT
|
|
2026-06-22 11:09:00 | INFO | Fetching Data from sql server for table-: Master_VisibilityReason ..............
|
|
2026-06-22 11:09:00 | INFO | Fetching data from sql server for Master table......
|
|
2026-06-22 11:09:00 | INFO | Fetching in progress ....
|
|
2026-06-22 11:09:01 | INFO | Fetched 17 rows from SQL Server
|
|
2026-06-22 11:09:01 | INFO | Fetched total row -: 17 from sql server for table-:Master_VisibilityReason ...........!!!
|
|
2026-06-22 11:09:01 | INFO | Fetched 17 rows
|
|
2026-06-22 11:09:01 | INFO | Truncated table Master_VisibilityReason
|
|
2026-06-22 11:09:01 | INFO | _ _ _ _Inserting data into clickhouse db from sql server_ _ _ _
|
|
2026-06-22 11:09:01 | INFO | Master_VisibilityReason: inserted 17 rows into ClickHouse
|
|
2026-06-22 11:09:01 | INFO | Master_VisibilityReason loaded successfully (17 rows)
|
|
2026-06-22 11:09:01 | INFO | ================================================================================
|
|
2026-06-22 11:09:01 | INFO | Processing Table: Master_VisibilityDefinition | Table type is -: DIMENSION | Based on -none and operation is used -DELETE+INSERT
|
|
2026-06-22 11:09:01 | INFO | Fetching Data from sql server for table-: Master_VisibilityDefinition ..............
|
|
2026-06-22 11:09:01 | INFO | Fetching data from sql server for Master table......
|
|
2026-06-22 11:09:01 | INFO | Fetching in progress ....
|
|
2026-06-22 11:09:02 | INFO | Fetched 861 rows from SQL Server
|
|
2026-06-22 11:09:02 | INFO | Fetched total row -: 861 from sql server for table-:Master_VisibilityDefinition ...........!!!
|
|
2026-06-22 11:09:02 | INFO | Fetched 861 rows
|
|
2026-06-22 11:09:02 | INFO | Truncated table Master_VisibilityDefinition
|
|
2026-06-22 11:09:02 | INFO | _ _ _ _Inserting data into clickhouse db from sql server_ _ _ _
|
|
2026-06-22 11:09:02 | INFO | Master_VisibilityDefinition: inserted 861 rows into ClickHouse
|
|
2026-06-22 11:09:02 | INFO | Master_VisibilityDefinition loaded successfully (861 rows)
|
|
2026-06-22 11:09:02 | INFO | ================================================================================
|
|
2026-06-22 11:09:02 | INFO | Processing Table: Master_Salesterritorylayer | Table type is -: DIMENSION | Based on -none and operation is used -DELETE+INSERT
|
|
2026-06-22 11:09:02 | INFO | Fetching Data from sql server for table-: Master_Salesterritorylayer ..............
|
|
2026-06-22 11:09:02 | INFO | Fetching data from sql server for Master table......
|
|
2026-06-22 11:09:02 | INFO | Fetching in progress ....
|
|
2026-06-22 11:09:02 | INFO | Fetched 33 rows from SQL Server
|
|
2026-06-22 11:09:02 | INFO | Fetched total row -: 33 from sql server for table-:Master_Salesterritorylayer ...........!!!
|
|
2026-06-22 11:09:02 | INFO | Fetched 33 rows
|
|
2026-06-22 11:09:02 | INFO | Truncated table Master_Salesterritorylayer
|
|
2026-06-22 11:09:02 | INFO | _ _ _ _Inserting data into clickhouse db from sql server_ _ _ _
|
|
2026-06-22 11:09:03 | INFO | Master_Salesterritorylayer: inserted 33 rows into ClickHouse
|
|
2026-06-22 11:09:03 | INFO | Master_Salesterritorylayer loaded successfully (33 rows)
|
|
2026-06-22 11:09:03 | INFO | ================================================================================
|
|
2026-06-22 11:09:03 | INFO | Pipeline Completed Successfully
|
|
2026-06-22 11:09:03 | INFO | ================================================================================
|
|
2026-06-22 12:04:40 | INFO | ================================================================================
|
|
2026-06-22 12:04:40 | INFO | Hello from data-move Python data pipeline!
|
|
2026-06-22 12:04:40 | INFO | Pipeline Run Date: 2026-06-21
|
|
2026-06-22 12:04:40 | INFO | Connecting to databases...
|
|
2026-06-22 12:04:43 | INFO | <sqlalchemy.engine.cursor.CursorResult object at 0x0000023A01CA4730>
|
|
2026-06-22 12:04:44 | INFO | <sqlalchemy.engine.cursor.CursorResult object at 0x0000023A02D1C7D0>
|
|
2026-06-22 12:04:45 | INFO | Database connections established
|
|
2026-06-22 12:04:45 | INFO | Collecting MIDs for: 2026-06-21
|
|
2026-06-22 12:04:46 | INFO | Found 569 MIDs
|
|
2026-06-22 12:04:48 | INFO | ================================================================================
|
|
2026-06-22 12:04:48 | INFO | Processing Table: Master_VisibilityReason | Table type is -: DIMENSION | Based on -none and operation is used -DELETE+INSERT
|
|
2026-06-22 12:04:48 | INFO | Fetching Data from sql server for table-: Master_VisibilityReason ..............
|
|
2026-06-22 12:04:48 | INFO | Fetching data from sql server for Master table......
|
|
2026-06-22 12:04:48 | INFO | Fetching in progress ....
|
|
2026-06-22 12:04:48 | INFO | Fetched 17 rows from SQL Server
|
|
2026-06-22 12:04:48 | INFO | Fetched total row -: 17 from sql server for table-:Master_VisibilityReason ...........!!!
|
|
2026-06-22 12:04:48 | INFO | Fetched 17 rows
|
|
2026-06-22 12:04:48 | INFO | Truncated table Master_VisibilityReason
|
|
2026-06-22 12:04:48 | INFO | _ _ _ _Inserting data into clickhouse db from sql server_ _ _ _
|
|
2026-06-22 12:04:48 | INFO | Master_VisibilityReason: inserted 17 rows into ClickHouse
|
|
2026-06-22 12:04:48 | INFO | Master_VisibilityReason loaded successfully (17 rows)
|
|
2026-06-22 12:04:48 | INFO | ================================================================================
|
|
2026-06-22 12:04:48 | INFO | Processing Table: Master_VisibilityDefinition | Table type is -: DIMENSION | Based on -none and operation is used -DELETE+INSERT
|
|
2026-06-22 12:04:48 | INFO | Fetching Data from sql server for table-: Master_VisibilityDefinition ..............
|
|
2026-06-22 12:04:48 | INFO | Fetching data from sql server for Master table......
|
|
2026-06-22 12:04:48 | INFO | Fetching in progress ....
|
|
2026-06-22 12:04:49 | INFO | Fetched 861 rows from SQL Server
|
|
2026-06-22 12:04:49 | INFO | Fetched total row -: 861 from sql server for table-:Master_VisibilityDefinition ...........!!!
|
|
2026-06-22 12:04:49 | INFO | Fetched 861 rows
|
|
2026-06-22 12:04:49 | INFO | Truncated table Master_VisibilityDefinition
|
|
2026-06-22 12:04:49 | INFO | _ _ _ _Inserting data into clickhouse db from sql server_ _ _ _
|
|
2026-06-22 12:04:49 | INFO | Master_VisibilityDefinition: inserted 861 rows into ClickHouse
|
|
2026-06-22 12:04:49 | INFO | Master_VisibilityDefinition loaded successfully (861 rows)
|
|
2026-06-22 12:04:49 | INFO | ================================================================================
|
|
2026-06-22 12:04:49 | INFO | Processing Table: Master_Salesterritorylayer | Table type is -: DIMENSION | Based on -none and operation is used -DELETE+INSERT
|
|
2026-06-22 12:04:49 | INFO | Fetching Data from sql server for table-: Master_Salesterritorylayer ..............
|
|
2026-06-22 12:04:49 | INFO | Fetching data from sql server for Master table......
|
|
2026-06-22 12:04:49 | INFO | Fetching in progress ....
|
|
2026-06-22 12:04:49 | INFO | Fetched 33 rows from SQL Server
|
|
2026-06-22 12:04:49 | INFO | Fetched total row -: 33 from sql server for table-:Master_Salesterritorylayer ...........!!!
|
|
2026-06-22 12:04:49 | INFO | Fetched 33 rows
|
|
2026-06-22 12:04:49 | INFO | Truncated table Master_Salesterritorylayer
|
|
2026-06-22 12:04:49 | INFO | _ _ _ _Inserting data into clickhouse db from sql server_ _ _ _
|
|
2026-06-22 12:04:50 | INFO | Master_Salesterritorylayer: inserted 33 rows into ClickHouse
|
|
2026-06-22 12:04:50 | INFO | Master_Salesterritorylayer loaded successfully (33 rows)
|
|
2026-06-22 12:04:50 | INFO | ================================================================================
|
|
2026-06-22 12:04:50 | INFO | Pipeline Completed Successfully
|
|
2026-06-22 12:04:50 | INFO | ================================================================================
|