Files
data_pipeline/logs/etl_20260612.log
T
Ankit Malik 8aaae1e27d 3rd commit
2026-06-12 15:39:39 +05:30

840 lines
68 KiB
Plaintext

2026-06-12 13:10:05 | INFO | Hello from data-move Python data pipeline !
2026-06-12 13:10:05 | INFO | connecting with both db servers sql-serveras well as clickhouse DB
2026-06-12 13:10:44 | INFO | Hello from data-move Python data pipeline !
2026-06-12 13:10:44 | INFO | connecting with both db servers sql-serveras well as clickhouse DB
2026-06-12 13:10:45 | INFO | <sqlalchemy.engine.cursor.CursorResult object at 0x0000016F8C267E30>
2026-06-12 13:10:50 | INFO | <sqlalchemy.engine.cursor.CursorResult object at 0x0000016F8D41F610>
2026-06-12 13:12:36 | INFO | Hello from data-move Python data pipeline !
2026-06-12 13:12:36 | INFO | connecting with both db servers sql-serveras well as clickhouse DB
2026-06-12 13:12:37 | INFO | <sqlalchemy.engine.cursor.CursorResult object at 0x000001FC11F47E30>
2026-06-12 13:12:39 | INFO | <sqlalchemy.engine.cursor.CursorResult object at 0x000001FC130E5D10>
2026-06-12 13:14:24 | INFO | Hello from data-move Python data pipeline !
2026-06-12 13:14:24 | INFO | connecting with both db servers sql-serveras well as clickhouse DB
2026-06-12 13:14:26 | INFO | <sqlalchemy.engine.cursor.CursorResult object at 0x000001E4C02ABE30>
2026-06-12 13:14:27 | INFO | <sqlalchemy.engine.cursor.CursorResult object at 0x000001E4C13D1D10>
2026-06-12 13:14:28 | INFO | Both databases connected successfully
2026-06-12 13:14:28 | INFO | Collecting MIDs for: 2026-06-11
2026-06-12 13:14:28 | INFO | Found 818 MIDs
2026-06-12 13:15:39 | INFO | Hello from data-move Python data pipeline !
2026-06-12 13:15:39 | INFO | connecting with both db servers sql-serveras well as clickhouse DB
2026-06-12 13:15:40 | INFO | <sqlalchemy.engine.cursor.CursorResult object at 0x000001C99BFFBE30>
2026-06-12 13:15:41 | INFO | <sqlalchemy.engine.cursor.CursorResult object at 0x000001C99D091D10>
2026-06-12 13:15:42 | INFO | Both databases connected successfully
2026-06-12 13:15:42 | INFO | Collecting MIDs for: 2026-06-11
2026-06-12 13:15:42 | INFO | Found 818 MIDs
2026-06-12 13:23:57 | INFO | Hello from data-move Python data pipeline !
2026-06-12 13:23:57 | INFO | connecting with both db servers sql-serveras well as clickhouse DB
2026-06-12 13:23:58 | INFO | <sqlalchemy.engine.cursor.CursorResult object at 0x000002369817BCD0>
2026-06-12 13:24:01 | INFO | <sqlalchemy.engine.cursor.CursorResult object at 0x000002369937DBD0>
2026-06-12 13:24:01 | INFO | Both databases connected successfully
2026-06-12 13:24:01 | INFO | Collecting MIDs for: 2026-06-11
2026-06-12 13:24:02 | INFO | Found 818 MIDs
2026-06-12 13:24:02 | INFO | Fetching data for 818 MIDs
2026-06-12 13:24:06 | INFO | Fetched 3,677 rows from SQL Server
2026-06-12 13:24:06 | INFO | Fetching OQaD data for 818 MIDs
2026-06-12 13:24:09 | INFO | Fetched 464 rows
2026-06-12 13:24:09 | INFO | Fetching Survey data for 818 MIDs
2026-06-12 13:24:10 | INFO | Fetched 142 Survey rows
2026-06-12 13:24:10 | INFO | Fetching Additional Visibility data for 818 MIDs
2026-06-12 13:24:11 | INFO | Fetched 1,922 Additional Visibility rows
2026-06-12 13:24:11 | INFO | Fetching coverage data for 818 MIDs
2026-06-12 13:24:11 | INFO | Fetched 761 rows from SQL Server
2026-06-12 13:24:11 | INFO | Fetching Login data for yesterday
2026-06-12 13:24:12 | INFO | Fetched 479 Login rows
2026-06-12 13:24:12 | INFO | Fetching Stock Details data for 818 MIDs
2026-06-12 13:24:17 | INFO | Fetched 41,628 Stock Details rows
2026-06-12 13:24:17 | INFO | Fetching Attendance data from 2026-05-27 to 2026-06-11
2026-06-12 13:24:18 | INFO | Fetched 11,918 attendance rows for 592 employees
2026-06-12 13:24:18 | INFO | Fetching Store Master data
2026-06-12 13:24:20 | INFO | Fetched 5,984 stores
2026-06-12 13:24:20 | INFO | Fetching SKU Master data
2026-06-12 13:24:20 | INFO | Fetched 160 SKU Master rows
2026-06-12 13:24:20 | INFO | Fetching Display Master data
2026-06-12 13:24:20 | INFO | Fetched 135 Display Master records
2026-06-12 13:24:20 | INFO | Fetching Employee Master data
2026-06-12 13:24:21 | INFO | Fetched 2,269 Employee Master records
2026-06-12 13:24:21 | INFO | Fetching Journey Plan for 2026-06
2026-06-12 13:24:21 | INFO | Fetched 20,969 Journey Plan records
2026-06-12 13:24:21 | INFO | Fetching Coverage Remarks
2026-06-12 13:24:21 | INFO | Fetched 29 Coverage Remark records
2026-06-12 13:24:21 | INFO | Fetching Mapping Store Visibility for 2026-06-11
2026-06-12 13:24:22 | INFO | Fetched 0 Mapping Store Visibility records
2026-06-12 13:24:22 | INFO | Fetching Master Visibility Reason data
2026-06-12 13:24:22 | INFO | Fetched 17 Master Visibility Reason records
2026-06-12 13:24:22 | INFO | Fetching Master Visibility Definition data
2026-06-12 13:24:22 | INFO | Fetched 861 Master Visibility Definition records
2026-06-12 13:24:22 | INFO | Fetching Web Login data for 2026-06-11
2026-06-12 13:24:22 | INFO | Fetched 223 Web Login records
2026-06-12 13:24:22 | INFO | Fetching Promotion data for 818 MIDs
2026-06-12 13:24:25 | INFO | Fetched 8,410 Promotion records
2026-06-12 13:26:34 | INFO | Hello from data-move Python data pipeline !
2026-06-12 13:26:34 | INFO | connecting with both db servers sql-serveras well as clickhouse DB
2026-06-12 13:26:34 | INFO | <sqlalchemy.engine.cursor.CursorResult object at 0x0000023042AABCD0>
2026-06-12 13:26:36 | INFO | <sqlalchemy.engine.cursor.CursorResult object at 0x0000023043C5DBD0>
2026-06-12 13:26:36 | INFO | Both databases connected successfully
2026-06-12 13:26:36 | INFO | Collecting MIDs for: 2026-06-11
2026-06-12 13:26:37 | INFO | Found 818 MIDs
2026-06-12 13:26:37 | INFO | Fetching data for 818 MIDs
2026-06-12 13:26:37 | INFO | Fetched 3,677 rows from SQL Server
2026-06-12 13:26:37 | INFO | Fetching OQaD data for 818 MIDs
2026-06-12 13:26:40 | INFO | Fetched 464 rows
2026-06-12 13:26:40 | INFO | Fetching Survey data for 818 MIDs
2026-06-12 13:26:40 | INFO | Fetched 142 Survey rows
2026-06-12 13:26:40 | INFO | Fetching Additional Visibility data for 818 MIDs
2026-06-12 13:26:41 | INFO | Fetched 1,922 Additional Visibility rows
2026-06-12 13:26:41 | INFO | Fetching coverage data for 818 MIDs
2026-06-12 13:26:41 | INFO | Fetched 761 rows from SQL Server
2026-06-12 13:26:41 | INFO | Fetching Login data for yesterday
2026-06-12 13:26:41 | INFO | Fetched 479 Login rows
2026-06-12 13:26:41 | INFO | Fetching Stock Details data for 818 MIDs
2026-06-12 13:26:45 | INFO | Fetched 41,628 Stock Details rows
2026-06-12 13:26:45 | INFO | Fetching Attendance data from 2026-05-27 to 2026-06-11
2026-06-12 13:26:46 | INFO | Fetched 11,918 attendance rows for 592 employees
2026-06-12 13:26:46 | INFO | Fetching Store Master data
2026-06-12 13:26:48 | INFO | Fetched 5,984 stores
2026-06-12 13:26:48 | INFO | Fetching SKU Master data
2026-06-12 13:26:48 | INFO | Fetched 160 SKU Master rows
2026-06-12 13:26:48 | INFO | Fetching Display Master data
2026-06-12 13:26:48 | INFO | Fetched 135 Display Master records
2026-06-12 13:26:48 | INFO | Fetching Employee Master data
2026-06-12 13:26:49 | INFO | Fetched 2,269 Employee Master records
2026-06-12 13:26:49 | INFO | Fetching Journey Plan for 2026-06
2026-06-12 13:26:50 | INFO | Fetched 20,969 Journey Plan records
2026-06-12 13:26:50 | INFO | Fetching Coverage Remarks
2026-06-12 13:26:50 | INFO | Fetched 29 Coverage Remark records
2026-06-12 13:26:50 | INFO | Fetching Mapping Store Visibility for 2026-06-11
2026-06-12 13:26:50 | INFO | Fetched 0 Mapping Store Visibility records
2026-06-12 13:26:50 | INFO | Fetching Master Visibility Reason data
2026-06-12 13:26:50 | INFO | Fetched 17 Master Visibility Reason records
2026-06-12 13:26:50 | INFO | Fetching Master Visibility Definition data
2026-06-12 13:26:50 | INFO | Fetched 861 Master Visibility Definition records
2026-06-12 13:26:50 | INFO | Fetching Web Login data for 2026-06-11
2026-06-12 13:26:50 | INFO | Fetched 223 Web Login records
2026-06-12 13:26:50 | INFO | Fetching Promotion data for 818 MIDs
2026-06-12 13:26:53 | INFO | Fetched 8,410 Promotion records
2026-06-12 13:29:18 | INFO | Hello from data-move Python data pipeline !
2026-06-12 13:29:18 | INFO | connecting with both db servers sql-serveras well as clickhouse DB
2026-06-12 13:29:19 | INFO | <sqlalchemy.engine.cursor.CursorResult object at 0x0000024FC853BCD0>
2026-06-12 13:29:21 | INFO | <sqlalchemy.engine.cursor.CursorResult object at 0x0000024FC971DBD0>
2026-06-12 13:29:21 | INFO | Both databases connected successfully
2026-06-12 13:29:21 | INFO | Collecting MIDs for: 2026-06-11
2026-06-12 13:29:21 | INFO | Found 818 MIDs
2026-06-12 13:29:21 | INFO | Fetching data for 818 MIDs
2026-06-12 13:29:22 | INFO | Fetched 3,677 rows from SQL Server
2026-06-12 13:29:22 | INFO | Fetching OQaD data for 818 MIDs
2026-06-12 13:29:24 | INFO | Fetched 464 rows
2026-06-12 13:29:24 | INFO | Fetching Survey data for 818 MIDs
2026-06-12 13:29:24 | INFO | Fetched 142 Survey rows
2026-06-12 13:29:24 | INFO | Fetching Additional Visibility data for 818 MIDs
2026-06-12 13:29:27 | INFO | Fetched 1,922 Additional Visibility rows
2026-06-12 13:29:27 | INFO | Fetching coverage data for 818 MIDs
2026-06-12 13:29:28 | INFO | Fetched 761 rows from SQL Server
2026-06-12 13:29:28 | INFO | Fetching Login data for yesterday
2026-06-12 13:29:28 | INFO | Fetched 479 Login rows
2026-06-12 13:29:28 | INFO | Fetching Stock Details data for 818 MIDs
2026-06-12 13:29:38 | INFO | Fetched 41,628 Stock Details rows
2026-06-12 13:29:38 | INFO | Fetching Attendance data from 2026-05-27 to 2026-06-11
2026-06-12 13:29:41 | INFO | Fetched 11,918 attendance rows for 592 employees
2026-06-12 13:29:41 | INFO | Fetching Store Master data
2026-06-12 13:29:46 | INFO | Fetched 5,984 stores
2026-06-12 13:29:46 | INFO | Fetching SKU Master data
2026-06-12 13:29:46 | INFO | Fetched 160 SKU Master rows
2026-06-12 13:29:46 | INFO | Fetching Display Master data
2026-06-12 13:29:46 | INFO | Fetched 135 Display Master records
2026-06-12 13:29:46 | INFO | Fetching Employee Master data
2026-06-12 13:29:47 | INFO | Fetched 2,269 Employee Master records
2026-06-12 13:29:47 | INFO | Fetching Journey Plan for 2026-06
2026-06-12 13:29:48 | INFO | Fetched 20,969 Journey Plan records
2026-06-12 13:29:48 | INFO | Fetching Coverage Remarks
2026-06-12 13:29:48 | INFO | Fetched 29 Coverage Remark records
2026-06-12 13:29:48 | INFO | Fetching Mapping Store Visibility for 2026-06-11
2026-06-12 13:29:48 | INFO | Fetched 0 Mapping Store Visibility records
2026-06-12 13:29:48 | INFO | Fetching Master Visibility Reason data
2026-06-12 13:29:48 | INFO | Fetched 17 Master Visibility Reason records
2026-06-12 13:29:48 | INFO | Fetching Master Visibility Definition data
2026-06-12 13:29:48 | INFO | Fetched 861 Master Visibility Definition records
2026-06-12 13:29:48 | INFO | Fetching Web Login data for 2026-06-11
2026-06-12 13:29:49 | INFO | Fetched 223 Web Login records
2026-06-12 13:29:49 | INFO | Fetching Promotion data for 818 MIDs
2026-06-12 13:29:55 | INFO | Fetched 8,410 Promotion records
2026-06-12 13:29:55 | INFO | Fetching Paid Visibility data for 818 MIDs
2026-06-12 13:30:05 | INFO | Fetched 1,098 Paid Visibility records
2026-06-12 13:30:05 | INFO | Fetching Master Sales Territory Layer data
2026-06-12 13:30:05 | INFO | Fetched 33 Master Sales Territory Layer records
2026-06-12 13:31:55 | INFO | Hello from data-move Python data pipeline !
2026-06-12 13:31:55 | INFO | connecting with both db servers sql-serveras well as clickhouse DB
2026-06-12 13:31:56 | INFO | <sqlalchemy.engine.cursor.CursorResult object at 0x000001B04B28BCD0>
2026-06-12 13:31:57 | INFO | <sqlalchemy.engine.cursor.CursorResult object at 0x000001B04C44DBD0>
2026-06-12 13:31:58 | INFO | Both databases connected successfully
2026-06-12 13:31:58 | INFO | Collecting MIDs for: 2026-06-11
2026-06-12 13:31:58 | INFO | Found 818 MIDs
2026-06-12 13:31:58 | INFO | Fetching data for 818 MIDs
2026-06-12 13:32:00 | INFO | Fetched 3,677 rows from SQL Server
2026-06-12 13:32:00 | INFO | Fetching OQaD data for 818 MIDs
2026-06-12 13:32:02 | INFO | Fetched 464 rows
2026-06-12 13:32:02 | INFO | Fetching Survey data for 818 MIDs
2026-06-12 13:32:02 | INFO | Fetched 142 Survey rows
2026-06-12 13:32:02 | INFO | Fetching Additional Visibility data for 818 MIDs
2026-06-12 13:32:04 | INFO | Fetched 1,922 Additional Visibility rows
2026-06-12 13:32:04 | INFO | Fetching coverage data for 818 MIDs
2026-06-12 13:32:04 | INFO | Fetched 761 rows from SQL Server
2026-06-12 13:32:04 | INFO | Fetching Login data for yesterday
2026-06-12 13:32:04 | INFO | Fetched 479 Login rows
2026-06-12 13:32:04 | INFO | Fetching Stock Details data for 818 MIDs
2026-06-12 13:32:08 | INFO | Fetched 41,628 Stock Details rows
2026-06-12 13:32:08 | INFO | Fetching Attendance data from 2026-05-27 to 2026-06-11
2026-06-12 13:32:09 | INFO | Fetched 11,918 attendance rows for 592 employees
2026-06-12 13:32:09 | INFO | Fetching Store Master data
2026-06-12 13:32:12 | INFO | Fetched 5,984 stores
2026-06-12 13:32:12 | INFO | Fetching SKU Master data
2026-06-12 13:32:12 | INFO | Fetched 160 SKU Master rows
2026-06-12 13:32:12 | INFO | Fetching Display Master data
2026-06-12 13:32:12 | INFO | Fetched 135 Display Master records
2026-06-12 13:32:12 | INFO | Fetching Employee Master data
2026-06-12 13:32:13 | INFO | Fetched 2,269 Employee Master records
2026-06-12 13:32:13 | INFO | Fetching Journey Plan for 2026-06
2026-06-12 13:32:13 | INFO | Fetched 20,969 Journey Plan records
2026-06-12 13:32:13 | INFO | Fetching Coverage Remarks
2026-06-12 13:32:13 | INFO | Fetched 29 Coverage Remark records
2026-06-12 13:32:13 | INFO | Fetching Mapping Store Visibility for 2026-06-11
2026-06-12 13:32:14 | INFO | Fetched 0 Mapping Store Visibility records
2026-06-12 13:32:14 | INFO | Fetching Master Visibility Reason data
2026-06-12 13:32:14 | INFO | Fetched 17 Master Visibility Reason records
2026-06-12 13:32:14 | INFO | Fetching Master Visibility Definition data
2026-06-12 13:32:14 | INFO | Fetched 861 Master Visibility Definition records
2026-06-12 13:32:14 | INFO | Fetching Web Login data for 2026-06-11
2026-06-12 13:32:14 | INFO | Fetched 223 Web Login records
2026-06-12 13:32:14 | INFO | Fetching Promotion data for 818 MIDs
2026-06-12 13:32:21 | INFO | Fetched 8,410 Promotion records
2026-06-12 13:32:21 | INFO | Fetching Paid Visibility data for 818 MIDs
2026-06-12 13:32:28 | INFO | Fetched 1,098 Paid Visibility records
2026-06-12 13:32:28 | INFO | Fetching Master Sales Territory Layer data
2026-06-12 13:32:29 | INFO | Fetched 33 Master Sales Territory Layer records
2026-06-12 13:33:34 | INFO | Hello from data-move Python data pipeline !
2026-06-12 13:33:34 | INFO | connecting with both db servers sql-serveras well as clickhouse DB
2026-06-12 13:33:34 | INFO | <sqlalchemy.engine.cursor.CursorResult object at 0x0000023647E2BCD0>
2026-06-12 13:33:36 | INFO | <sqlalchemy.engine.cursor.CursorResult object at 0x0000023648FDDBD0>
2026-06-12 13:33:36 | INFO | Both databases connected successfully
2026-06-12 13:33:36 | INFO | Collecting MIDs for: 2026-06-11
2026-06-12 13:33:36 | INFO | Found 818 MIDs
2026-06-12 13:33:37 | INFO | Fetching data for 818 MIDs
2026-06-12 13:33:37 | INFO | Fetched 3,677 rows from SQL Server
2026-06-12 13:33:37 | INFO | Fetching OQaD data for 818 MIDs
2026-06-12 13:33:39 | INFO | Fetched 464 rows
2026-06-12 13:33:39 | INFO | Fetching Survey data for 818 MIDs
2026-06-12 13:33:40 | INFO | Fetched 142 Survey rows
2026-06-12 13:33:40 | INFO | Fetching Additional Visibility data for 818 MIDs
2026-06-12 13:35:57 | INFO | Hello from data-move Python data pipeline !
2026-06-12 13:35:57 | INFO | connecting with both db servers sql-serveras well as clickhouse DB
2026-06-12 13:35:57 | INFO | <sqlalchemy.engine.cursor.CursorResult object at 0x000002D27D8DBCD0>
2026-06-12 13:35:59 | INFO | <sqlalchemy.engine.cursor.CursorResult object at 0x000002D27EA9DBD0>
2026-06-12 13:35:59 | INFO | Both databases connected successfully
2026-06-12 13:35:59 | INFO | Collecting MIDs for: 2026-06-11
2026-06-12 13:36:00 | INFO | Found 818 MIDs
2026-06-12 13:36:00 | INFO | ================================================================================
2026-06-12 13:36:00 | INFO | TABLE=SOS_OneApp | TYPE=FACT | OPERATION=INSERT
2026-06-12 13:36:00 | INFO | Fetching data for 818 MIDs
2026-06-12 13:36:00 | INFO | Fetched 3,677 rows from SQL Server
2026-06-12 13:36:00 | INFO | ================================================================================
2026-06-12 13:36:00 | INFO | TABLE=OQaD | TYPE=FACT | OPERATION=INSERT
2026-06-12 13:36:00 | INFO | Fetching OQaD data for 818 MIDs
2026-06-12 13:36:03 | INFO | Fetched 464 rows
2026-06-12 13:36:03 | INFO | ================================================================================
2026-06-12 13:36:03 | INFO | TABLE=Survey | TYPE=FACT | OPERATION=INSERT
2026-06-12 13:36:03 | INFO | Fetching Survey data for 818 MIDs
2026-06-12 13:36:03 | INFO | Fetched 142 Survey rows
2026-06-12 13:36:03 | INFO | ================================================================================
2026-06-12 13:36:03 | INFO | TABLE=additional_visibility | TYPE=FACT | OPERATION=INSERT
2026-06-12 13:36:03 | INFO | Fetching Additional Visibility data for 818 MIDs
2026-06-12 13:36:04 | INFO | Fetched 1,922 Additional Visibility rows
2026-06-12 13:36:04 | INFO | ================================================================================
2026-06-12 13:36:04 | INFO | TABLE=Coverage | TYPE=FACT | OPERATION=INSERT
2026-06-12 13:36:04 | INFO | Fetching coverage data for 818 MIDs
2026-06-12 13:36:04 | INFO | Fetched 761 rows from SQL Server
2026-06-12 13:36:04 | INFO | ================================================================================
2026-06-12 13:36:04 | INFO | TABLE=Login | TYPE=FACT | OPERATION=INSERT
2026-06-12 13:36:04 | INFO | Fetching Login data for yesterday
2026-06-12 13:36:04 | INFO | Fetched 479 Login rows
2026-06-12 13:36:04 | INFO | ================================================================================
2026-06-12 13:36:04 | INFO | TABLE=Stock_Details | TYPE=FACT | OPERATION=INSERT
2026-06-12 13:36:04 | INFO | Fetching Stock Details data for 818 MIDs
2026-06-12 13:36:08 | INFO | Fetched 41,628 Stock Details rows
2026-06-12 13:36:08 | INFO | ================================================================================
2026-06-12 13:36:08 | INFO | TABLE=Attendance | TYPE=FACT | OPERATION=DELETE+INSERT
2026-06-12 13:36:08 | INFO | Fetching Attendance data from 2026-05-27 to 2026-06-11
2026-06-12 13:36:13 | INFO | Fetched 11,918 attendance rows for 592 employees
2026-06-12 13:36:13 | INFO | ================================================================================
2026-06-12 13:36:13 | INFO | TABLE=Store_Master | TYPE=DIMENSION | OPERATION=DELETE+INSERT
2026-06-12 13:36:13 | INFO | Fetching Store Master data
2026-06-12 13:36:17 | INFO | Fetched 5,984 stores
2026-06-12 13:36:17 | INFO | ================================================================================
2026-06-12 13:36:17 | INFO | TABLE=SKU_Master | TYPE=DIMENSION | OPERATION=DELETE+INSERT
2026-06-12 13:36:17 | INFO | Fetching SKU Master data
2026-06-12 13:36:17 | INFO | Fetched 160 SKU Master rows
2026-06-12 13:36:17 | INFO | ================================================================================
2026-06-12 13:36:17 | INFO | TABLE=display_master | TYPE=DIMENSION | OPERATION=DELETE+INSERT
2026-06-12 13:36:17 | INFO | Fetching Display Master data
2026-06-12 13:36:17 | INFO | Fetched 135 Display Master records
2026-06-12 13:36:17 | INFO | ================================================================================
2026-06-12 13:36:17 | INFO | TABLE=Employee_Master | TYPE=DIMENSION | OPERATION=DELETE+INSERT
2026-06-12 13:36:17 | INFO | Fetching Employee Master data
2026-06-12 13:36:18 | INFO | Fetched 2,269 Employee Master records
2026-06-12 13:36:18 | INFO | ================================================================================
2026-06-12 13:36:18 | INFO | TABLE=Journey_Plan | TYPE=FACT | OPERATION=INSERT
2026-06-12 13:36:18 | INFO | Fetching Journey Plan for 2026-06
2026-06-12 13:36:18 | INFO | Fetched 20,970 Journey Plan records
2026-06-12 13:36:18 | INFO | ================================================================================
2026-06-12 13:36:18 | INFO | TABLE=coverage_remarks | TYPE=FACT | OPERATION=INSERT
2026-06-12 13:36:18 | INFO | Fetching Coverage Remarks
2026-06-12 13:36:18 | INFO | Fetched 29 Coverage Remark records
2026-06-12 13:36:18 | INFO | ================================================================================
2026-06-12 13:36:18 | INFO | TABLE=mapping_storevisibility | TYPE=BRIDGE | OPERATION=INSERT
2026-06-12 13:36:18 | INFO | Fetching Mapping Store Visibility for 2026-06-11
2026-06-12 13:36:18 | INFO | Fetched 0 Mapping Store Visibility records
2026-06-12 13:36:18 | INFO | ================================================================================
2026-06-12 13:36:18 | INFO | TABLE=Master_VisibilityReason | TYPE=DIMENSION | OPERATION=INSERT
2026-06-12 13:36:18 | INFO | Fetching Master Visibility Reason data
2026-06-12 13:36:19 | INFO | Fetched 17 Master Visibility Reason records
2026-06-12 13:36:19 | INFO | ================================================================================
2026-06-12 13:36:19 | INFO | TABLE=Master_VisibilityDefinition | TYPE=DIMENSION | OPERATION=INSERT
2026-06-12 13:36:19 | INFO | Fetching Master Visibility Definition data
2026-06-12 13:36:19 | INFO | Fetched 861 Master Visibility Definition records
2026-06-12 13:36:19 | INFO | ================================================================================
2026-06-12 13:36:19 | INFO | TABLE=Web_Logins | TYPE=FACT | OPERATION=INSERT
2026-06-12 13:36:19 | INFO | Fetching Web Login data for 2026-06-11
2026-06-12 13:36:19 | INFO | Fetched 223 Web Login records
2026-06-12 13:36:19 | INFO | ================================================================================
2026-06-12 13:36:19 | INFO | TABLE=Promotion | TYPE=FACT | OPERATION=INSERT
2026-06-12 13:36:19 | INFO | Fetching Promotion data for 818 MIDs
2026-06-12 13:36:22 | INFO | Fetched 8,410 Promotion records
2026-06-12 13:36:22 | INFO | ================================================================================
2026-06-12 13:36:22 | INFO | TABLE=PaidVisibility | TYPE=FACT | OPERATION=INSERT
2026-06-12 13:36:22 | INFO | Fetching Paid Visibility data for 818 MIDs
2026-06-12 13:36:30 | INFO | Fetched 1,098 Paid Visibility records
2026-06-12 13:36:30 | INFO | ================================================================================
2026-06-12 13:36:30 | INFO | TABLE=Master_Salesterritorylayer | TYPE=DIMENSION | OPERATION=INSERT
2026-06-12 13:36:30 | INFO | Fetching Master Sales Territory Layer data
2026-06-12 13:36:31 | INFO | Fetched 33 Master Sales Territory Layer records
2026-06-12 13:40:32 | INFO | Hello from data-move Python data pipeline !
2026-06-12 13:40:32 | INFO | connecting with both db servers sql-serveras well as clickhouse DB
2026-06-12 13:40:32 | INFO | <sqlalchemy.engine.cursor.CursorResult object at 0x000002433A46BCD0>
2026-06-12 13:40:34 | INFO | <sqlalchemy.engine.cursor.CursorResult object at 0x000002433B61DBD0>
2026-06-12 13:40:35 | INFO | Both databases connected successfully
2026-06-12 13:40:35 | INFO | Collecting MIDs for: 2026-06-11
2026-06-12 13:40:35 | INFO | Found 818 MIDs
2026-06-12 13:40:35 | INFO | ================================================================================
2026-06-12 13:40:35 | INFO | TABLE=SOS_OneApp | TYPE=FACT | OPERATION=INSERT
2026-06-12 13:40:35 | INFO | Fetching data for 818 MIDs
2026-06-12 13:40:36 | INFO | Fetched 3,677 rows from SQL Server
2026-06-12 13:40:36 | INFO | ================================================================================
2026-06-12 13:40:36 | INFO | TABLE=OQaD | TYPE=FACT | OPERATION=INSERT
2026-06-12 13:40:36 | INFO | Fetching OQaD data for 818 MIDs
2026-06-12 13:40:38 | INFO | Fetched 464 rows
2026-06-12 13:40:38 | INFO | ================================================================================
2026-06-12 13:40:38 | INFO | TABLE=Survey | TYPE=FACT | OPERATION=INSERT
2026-06-12 13:40:38 | INFO | Fetching Survey data for 818 MIDs
2026-06-12 13:40:38 | INFO | Fetched 142 Survey rows
2026-06-12 13:40:38 | INFO | ================================================================================
2026-06-12 13:40:38 | INFO | TABLE=additional_visibility | TYPE=FACT | OPERATION=INSERT
2026-06-12 13:40:38 | INFO | Fetching Additional Visibility data for 818 MIDs
2026-06-12 13:40:39 | INFO | Fetched 1,922 Additional Visibility rows
2026-06-12 13:40:39 | INFO | ================================================================================
2026-06-12 13:40:39 | INFO | TABLE=Coverage | TYPE=FACT | OPERATION=INSERT
2026-06-12 13:40:39 | INFO | Fetching coverage data for 818 MIDs
2026-06-12 13:40:39 | INFO | Fetched 761 rows from SQL Server
2026-06-12 13:40:39 | INFO | ================================================================================
2026-06-12 13:40:39 | INFO | TABLE=Login | TYPE=FACT | OPERATION=INSERT
2026-06-12 13:40:39 | INFO | Fetching Login data for yesterday
2026-06-12 13:40:39 | INFO | Fetched 479 Login rows
2026-06-12 13:40:39 | INFO | ================================================================================
2026-06-12 13:40:39 | INFO | TABLE=Stock_Details | TYPE=FACT | OPERATION=INSERT
2026-06-12 13:40:39 | INFO | Fetching Stock Details data for 818 MIDs
2026-06-12 13:40:41 | INFO | Fetched 41,628 Stock Details rows
2026-06-12 13:40:41 | INFO | ================================================================================
2026-06-12 13:40:41 | INFO | TABLE=Attendance | TYPE=FACT | OPERATION=DELETE+INSERT
2026-06-12 13:40:41 | INFO | Fetching Attendance data from 2026-05-27 to 2026-06-11
2026-06-12 13:40:42 | INFO | Fetched 11,918 attendance rows for 592 employees
2026-06-12 13:40:42 | INFO | ================================================================================
2026-06-12 13:40:42 | INFO | TABLE=Store_Master | TYPE=DIMENSION | OPERATION=DELETE+INSERT
2026-06-12 13:40:42 | INFO | Fetching Store Master data
2026-06-12 13:40:44 | INFO | Fetched 5,984 stores
2026-06-12 13:40:44 | INFO | ================================================================================
2026-06-12 13:40:44 | INFO | TABLE=SKU_Master | TYPE=DIMENSION | OPERATION=DELETE+INSERT
2026-06-12 13:40:44 | INFO | Fetching SKU Master data
2026-06-12 13:40:44 | INFO | Fetched 160 SKU Master rows
2026-06-12 13:40:44 | INFO | ================================================================================
2026-06-12 13:40:44 | INFO | TABLE=display_master | TYPE=DIMENSION | OPERATION=DELETE+INSERT
2026-06-12 13:40:44 | INFO | Fetching Display Master data
2026-06-12 13:40:44 | INFO | Fetched 135 Display Master records
2026-06-12 13:40:44 | INFO | ================================================================================
2026-06-12 13:40:44 | INFO | TABLE=Employee_Master | TYPE=DIMENSION | OPERATION=DELETE+INSERT
2026-06-12 13:40:44 | INFO | Fetching Employee Master data
2026-06-12 13:40:44 | INFO | Fetched 2,269 Employee Master records
2026-06-12 13:40:44 | INFO | ================================================================================
2026-06-12 13:40:44 | INFO | TABLE=Journey_Plan | TYPE=FACT | OPERATION=INSERT
2026-06-12 13:40:44 | INFO | Fetching Journey Plan for 2026-06
2026-06-12 13:40:45 | INFO | Fetched 20,972 Journey Plan records
2026-06-12 13:40:45 | INFO | ================================================================================
2026-06-12 13:40:45 | INFO | TABLE=coverage_remarks | TYPE=FACT | OPERATION=INSERT
2026-06-12 13:40:45 | INFO | Fetching Coverage Remarks
2026-06-12 13:40:45 | INFO | Fetched 29 Coverage Remark records
2026-06-12 13:40:45 | INFO | ================================================================================
2026-06-12 13:40:45 | INFO | TABLE=mapping_storevisibility | TYPE=BRIDGE | OPERATION=INSERT
2026-06-12 13:40:45 | INFO | Fetching Mapping Store Visibility for 2026-06-11
2026-06-12 13:40:45 | INFO | Fetched 0 Mapping Store Visibility records
2026-06-12 13:40:45 | INFO | ================================================================================
2026-06-12 13:40:45 | INFO | TABLE=Master_VisibilityReason | TYPE=DIMENSION | OPERATION=INSERT
2026-06-12 13:40:45 | INFO | Fetching Master Visibility Reason data
2026-06-12 13:40:45 | INFO | Fetched 17 Master Visibility Reason records
2026-06-12 13:40:45 | INFO | ================================================================================
2026-06-12 13:40:45 | INFO | TABLE=Master_VisibilityDefinition | TYPE=DIMENSION | OPERATION=INSERT
2026-06-12 13:40:45 | INFO | Fetching Master Visibility Definition data
2026-06-12 13:40:45 | INFO | Fetched 861 Master Visibility Definition records
2026-06-12 13:40:45 | INFO | ================================================================================
2026-06-12 13:40:45 | INFO | TABLE=Web_Logins | TYPE=FACT | OPERATION=INSERT
2026-06-12 13:40:45 | INFO | Fetching Web Login data for 2026-06-11
2026-06-12 13:40:45 | INFO | Fetched 223 Web Login records
2026-06-12 13:40:45 | INFO | ================================================================================
2026-06-12 13:40:45 | INFO | TABLE=Promotion | TYPE=FACT | OPERATION=INSERT
2026-06-12 13:40:45 | INFO | Fetching Promotion data for 818 MIDs
2026-06-12 13:40:48 | INFO | Fetched 8,410 Promotion records
2026-06-12 13:40:48 | INFO | ================================================================================
2026-06-12 13:40:48 | INFO | TABLE=PaidVisibility | TYPE=FACT | OPERATION=INSERT
2026-06-12 13:40:48 | INFO | Fetching Paid Visibility data for 818 MIDs
2026-06-12 13:40:55 | INFO | Fetched 1,098 Paid Visibility records
2026-06-12 13:40:55 | INFO | ================================================================================
2026-06-12 13:40:55 | INFO | TABLE=Master_Salesterritorylayer | TYPE=DIMENSION | OPERATION=INSERT
2026-06-12 13:40:55 | INFO | Fetching Master Sales Territory Layer data
2026-06-12 13:40:55 | INFO | Fetched 33 Master Sales Territory Layer records
2026-06-12 13:45:23 | INFO | Hello from data-move Python data pipeline !
2026-06-12 13:45:23 | INFO | connecting with both db servers sql-serveras well as clickhouse DB
2026-06-12 13:45:24 | INFO | <sqlalchemy.engine.cursor.CursorResult object at 0x000001FC647ABCD0>
2026-06-12 13:45:26 | INFO | <sqlalchemy.engine.cursor.CursorResult object at 0x000001FC65961BD0>
2026-06-12 13:45:26 | INFO | Both databases connected successfully
2026-06-12 13:45:26 | INFO | Collecting MIDs for: 2026-06-11
2026-06-12 13:45:26 | INFO | Found 818 MIDs
2026-06-12 13:45:26 | INFO | ================================================================================
2026-06-12 13:45:26 | INFO | TABLE=SOS_OneApp | TYPE=FACT | OPERATION=INSERT
2026-06-12 13:45:26 | INFO | Fetching data for 818 MIDs
2026-06-12 13:45:27 | INFO | Fetched 3,677 rows from SQL Server
2026-06-12 13:45:27 | INFO | ================================================================================
2026-06-12 13:45:27 | INFO | TABLE=OQaD | TYPE=FACT | OPERATION=INSERT
2026-06-12 13:45:27 | INFO | Fetching OQaD data for 818 MIDs
2026-06-12 13:45:29 | INFO | Fetched 464 rows
2026-06-12 13:45:29 | INFO | ================================================================================
2026-06-12 13:45:29 | INFO | TABLE=Survey | TYPE=FACT | OPERATION=INSERT
2026-06-12 13:45:29 | INFO | Fetching Survey data for 818 MIDs
2026-06-12 13:45:30 | INFO | Fetched 142 Survey rows
2026-06-12 13:45:30 | INFO | ================================================================================
2026-06-12 13:45:30 | INFO | TABLE=additional_visibility | TYPE=FACT | OPERATION=INSERT
2026-06-12 13:45:30 | INFO | Fetching Additional Visibility data for 818 MIDs
2026-06-12 13:45:31 | INFO | Fetched 1,922 Additional Visibility rows
2026-06-12 13:45:31 | INFO | ================================================================================
2026-06-12 13:45:31 | INFO | TABLE=Coverage | TYPE=FACT | OPERATION=INSERT
2026-06-12 13:45:31 | INFO | Fetching coverage data for 818 MIDs
2026-06-12 13:45:31 | INFO | Fetched 761 rows from SQL Server
2026-06-12 13:45:31 | INFO | ================================================================================
2026-06-12 13:45:31 | INFO | TABLE=Login | TYPE=FACT | OPERATION=INSERT
2026-06-12 13:45:31 | INFO | Fetching Login data for yesterday
2026-06-12 13:45:31 | INFO | Fetched 479 Login rows
2026-06-12 13:45:31 | INFO | ================================================================================
2026-06-12 13:45:31 | INFO | TABLE=Stock_Details | TYPE=FACT | OPERATION=INSERT
2026-06-12 13:45:31 | INFO | Fetching Stock Details data for 818 MIDs
2026-06-12 13:45:34 | INFO | Fetched 41,628 Stock Details rows
2026-06-12 13:45:34 | INFO | ================================================================================
2026-06-12 13:45:34 | INFO | TABLE=Attendance | TYPE=FACT | OPERATION=DELETE+INSERT
2026-06-12 13:45:34 | INFO | Fetching Attendance data from 2026-05-27 to 2026-06-11
2026-06-12 13:45:35 | INFO | Fetched 11,918 attendance rows for 592 employees
2026-06-12 13:45:35 | INFO | ================================================================================
2026-06-12 13:45:35 | INFO | TABLE=Store_Master | TYPE=DIMENSION | OPERATION=DELETE+INSERT
2026-06-12 13:45:35 | INFO | ================================================================================
2026-06-12 13:45:35 | INFO | Fetching Store Master data
2026-06-12 13:45:37 | INFO | ================================================================================
2026-06-12 13:45:37 | INFO | Fetched 5,984 stores
2026-06-12 13:45:37 | INFO | ================================================================================
2026-06-12 13:45:37 | INFO | TABLE=SKU_Master | TYPE=DIMENSION | OPERATION=DELETE+INSERT
2026-06-12 13:45:37 | INFO | ================================================================================
2026-06-12 13:45:37 | INFO | Fetching SKU Master data
2026-06-12 13:45:37 | INFO | ================================================================================
2026-06-12 13:45:37 | INFO | Fetched 160 SKU Master rows
2026-06-12 13:45:37 | INFO | ================================================================================
2026-06-12 13:45:37 | INFO | TABLE=display_master | TYPE=DIMENSION | OPERATION=DELETE+INSERT
2026-06-12 13:45:37 | INFO | ================================================================================
2026-06-12 13:45:37 | INFO | Fetching Display Master data
2026-06-12 13:45:37 | INFO | ================================================================================
2026-06-12 13:45:37 | INFO | Fetched 135 Display Master records
2026-06-12 13:45:37 | INFO | ================================================================================
2026-06-12 13:45:37 | INFO | TABLE=Employee_Master | TYPE=DIMENSION | OPERATION=DELETE+INSERT
2026-06-12 13:45:37 | INFO | ================================================================================
2026-06-12 13:45:37 | INFO | Fetching Employee Master data
2026-06-12 13:45:38 | INFO | ================================================================================
2026-06-12 13:45:38 | INFO | Fetched 2,269 Employee Master records
2026-06-12 13:45:38 | INFO | ================================================================================
2026-06-12 13:45:38 | INFO | TABLE=Journey_Plan | TYPE=FACT | OPERATION=INSERT
2026-06-12 13:45:38 | INFO | Fetching Journey Plan for 2026-06
2026-06-12 13:45:38 | INFO | Fetched 20,972 Journey Plan records
2026-06-12 13:45:38 | INFO | ================================================================================
2026-06-12 13:45:38 | INFO | TABLE=coverage_remarks | TYPE=FACT | OPERATION=INSERT
2026-06-12 13:45:38 | INFO | Fetching Coverage Remarks
2026-06-12 13:45:38 | INFO | Fetched 29 Coverage Remark records
2026-06-12 13:45:38 | INFO | ================================================================================
2026-06-12 13:45:38 | INFO | TABLE=mapping_storevisibility | TYPE=BRIDGE | OPERATION=INSERT
2026-06-12 13:45:38 | INFO | Fetching Mapping Store Visibility for 2026-06-11
2026-06-12 13:45:38 | INFO | Fetched 0 Mapping Store Visibility records
2026-06-12 13:45:38 | INFO | ================================================================================
2026-06-12 13:45:38 | INFO | TABLE=Master_VisibilityReason | TYPE=DIMENSION | OPERATION=INSERT
2026-06-12 13:45:38 | INFO | ================================================================================
2026-06-12 13:45:38 | INFO | Fetching Master Visibility Reason data
2026-06-12 13:45:38 | INFO | ================================================================================
2026-06-12 13:45:38 | INFO | Fetched 17 Master Visibility Reason records
2026-06-12 13:45:38 | INFO | ================================================================================
2026-06-12 13:45:38 | INFO | TABLE=Master_VisibilityDefinition | TYPE=DIMENSION | OPERATION=INSERT
2026-06-12 13:45:38 | INFO | ================================================================================
2026-06-12 13:45:38 | INFO | Fetching Master Visibility Definition data
2026-06-12 13:45:38 | INFO | ================================================================================
2026-06-12 13:45:38 | INFO | Fetched 861 Master Visibility Definition records
2026-06-12 13:45:38 | INFO | ================================================================================
2026-06-12 13:45:38 | INFO | TABLE=Web_Logins | TYPE=FACT | OPERATION=INSERT
2026-06-12 13:45:38 | INFO | Fetching Web Login data for 2026-06-11
2026-06-12 13:45:39 | INFO | Fetched 223 Web Login records
2026-06-12 13:45:39 | INFO | ================================================================================
2026-06-12 13:45:39 | INFO | TABLE=Promotion | TYPE=FACT | OPERATION=INSERT
2026-06-12 13:45:39 | INFO | Fetching Promotion data for 818 MIDs
2026-06-12 13:45:42 | INFO | Fetched 8,410 Promotion records
2026-06-12 13:45:42 | INFO | ================================================================================
2026-06-12 13:45:42 | INFO | TABLE=PaidVisibility | TYPE=FACT | OPERATION=INSERT
2026-06-12 13:45:42 | INFO | Fetching Paid Visibility data for 818 MIDs
2026-06-12 13:45:49 | INFO | Fetched 1,098 Paid Visibility records
2026-06-12 13:45:49 | INFO | ================================================================================
2026-06-12 13:45:50 | INFO | TABLE=Master_Salesterritorylayer | TYPE=DIMENSION | OPERATION=INSERT
2026-06-12 13:45:50 | INFO | ================================================================================
2026-06-12 13:45:50 | INFO | Fetching Master Sales Territory Layer data
2026-06-12 13:45:50 | INFO | ================================================================================
2026-06-12 13:45:50 | INFO | Fetched 33 Master Sales Territory Layer records
2026-06-12 13:48:24 | INFO | Hello from data-move Python data pipeline !
2026-06-12 13:48:24 | INFO | ================================================================================
2026-06-12 13:48:24 | INFO | connecting with both db servers sql-serveras well as clickhouse DB
2026-06-12 13:48:24 | INFO | ================================================================================
2026-06-12 13:48:24 | INFO | <sqlalchemy.engine.cursor.CursorResult object at 0x00000252AB9FBCD0>
2026-06-12 13:48:26 | INFO | ================================================================================
2026-06-12 13:48:26 | INFO | <sqlalchemy.engine.cursor.CursorResult object at 0x00000252ACBB9E50>
2026-06-12 13:48:26 | INFO | ================================================================================
2026-06-12 13:48:26 | INFO | Both databases connected successfully
2026-06-12 13:48:26 | INFO | ================================================================================
2026-06-12 13:48:26 | INFO | Collecting MIDs for: 2026-06-11
2026-06-12 13:48:27 | INFO | ================================================================================
2026-06-12 13:48:27 | INFO | Found 818 MIDs
2026-06-12 13:48:27 | INFO | ================================================================================
2026-06-12 13:48:27 | INFO | TABLE=SOS_OneApp | TYPE=FACT | OPERATION=INSERT
2026-06-12 13:48:27 | INFO | ================================================================================
2026-06-12 13:48:27 | INFO | Fetching data for 818 MIDs
2026-06-12 13:48:27 | INFO | ================================================================================
2026-06-12 13:48:27 | INFO | Fetched 3,677 rows from SQL Server
2026-06-12 13:48:27 | INFO | ================================================================================
2026-06-12 13:48:27 | INFO | TABLE=OQaD | TYPE=FACT | OPERATION=INSERT
2026-06-12 13:48:27 | INFO | ================================================================================
2026-06-12 13:48:27 | INFO | Fetching OQaD data for 818 MIDs
2026-06-12 13:48:29 | INFO | ================================================================================
2026-06-12 13:48:29 | INFO | Fetched 464 rows
2026-06-12 13:48:29 | INFO | ================================================================================
2026-06-12 13:48:29 | INFO | TABLE=Survey | TYPE=FACT | OPERATION=INSERT
2026-06-12 13:48:29 | INFO | ================================================================================
2026-06-12 13:48:29 | INFO | Fetching Survey data for 818 MIDs
2026-06-12 13:48:30 | INFO | ================================================================================
2026-06-12 13:48:30 | INFO | Fetched 142 Survey rows
2026-06-12 13:48:30 | INFO | ================================================================================
2026-06-12 13:48:30 | INFO | TABLE=additional_visibility | TYPE=FACT | OPERATION=INSERT
2026-06-12 13:48:30 | INFO | ================================================================================
2026-06-12 13:48:30 | INFO | Fetching Additional Visibility data for 818 MIDs
2026-06-12 13:48:30 | INFO | ================================================================================
2026-06-12 13:48:30 | INFO | Fetched 1,922 Additional Visibility rows
2026-06-12 13:48:30 | INFO | ================================================================================
2026-06-12 13:48:30 | INFO | TABLE=Coverage | TYPE=FACT | OPERATION=INSERT
2026-06-12 13:48:30 | INFO | ================================================================================
2026-06-12 13:48:30 | INFO | Fetching coverage data for 818 MIDs
2026-06-12 13:48:30 | INFO | ================================================================================
2026-06-12 13:48:30 | INFO | Fetched 761 rows from SQL Server
2026-06-12 13:48:30 | INFO | ================================================================================
2026-06-12 13:48:30 | INFO | TABLE=Login | TYPE=FACT | OPERATION=INSERT
2026-06-12 13:48:30 | INFO | ================================================================================
2026-06-12 13:48:30 | INFO | Fetching Login data for yesterday
2026-06-12 13:48:31 | INFO | ================================================================================
2026-06-12 13:48:31 | INFO | Fetched 479 Login rows
2026-06-12 13:48:31 | INFO | ================================================================================
2026-06-12 13:48:31 | INFO | TABLE=Stock_Details | TYPE=FACT | OPERATION=INSERT
2026-06-12 13:48:31 | INFO | ================================================================================
2026-06-12 13:48:31 | INFO | Fetching Stock Details data for 818 MIDs
2026-06-12 13:48:34 | INFO | Fetched 41,628 Stock Details rows
2026-06-12 13:48:34 | INFO | ================================================================================
2026-06-12 13:48:34 | INFO | TABLE=Attendance | TYPE=FACT | OPERATION=DELETE+INSERT
2026-06-12 13:48:34 | INFO | Fetching Attendance data from 2026-05-27 to 2026-06-11
2026-06-12 13:48:35 | INFO | Fetched 11,918 attendance rows for 592 employees
2026-06-12 13:48:35 | INFO | ================================================================================
2026-06-12 13:48:35 | INFO | TABLE=Store_Master | TYPE=DIMENSION | OPERATION=DELETE+INSERT
2026-06-12 13:48:35 | INFO | Fetching Store Master data
2026-06-12 13:48:37 | INFO | Fetched 5,984 stores
2026-06-12 13:48:37 | INFO | ================================================================================
2026-06-12 13:48:37 | INFO | TABLE=SKU_Master | TYPE=DIMENSION | OPERATION=DELETE+INSERT
2026-06-12 13:48:37 | INFO | Fetching SKU Master data
2026-06-12 13:48:37 | INFO | Fetched 160 SKU Master rows
2026-06-12 13:48:37 | INFO | ================================================================================
2026-06-12 13:48:37 | INFO | TABLE=display_master | TYPE=DIMENSION | OPERATION=DELETE+INSERT
2026-06-12 13:48:37 | INFO | Fetching Display Master data
2026-06-12 13:48:37 | INFO | Fetched 135 Display Master records
2026-06-12 13:48:37 | INFO | ================================================================================
2026-06-12 13:48:37 | INFO | TABLE=Employee_Master | TYPE=DIMENSION | OPERATION=DELETE+INSERT
2026-06-12 13:48:37 | INFO | Fetching Employee Master data
2026-06-12 13:48:37 | INFO | Fetched 2,269 Employee Master records
2026-06-12 13:48:37 | INFO | ================================================================================
2026-06-12 13:48:37 | INFO | TABLE=Journey_Plan | TYPE=FACT | OPERATION=INSERT
2026-06-12 13:48:37 | INFO | Fetching Journey Plan for 2026-06
2026-06-12 13:48:38 | INFO | Fetched 20,973 Journey Plan records
2026-06-12 13:48:38 | INFO | ================================================================================
2026-06-12 13:48:38 | INFO | TABLE=coverage_remarks | TYPE=FACT | OPERATION=INSERT
2026-06-12 13:48:38 | INFO | Fetching Coverage Remarks
2026-06-12 13:48:38 | INFO | Fetched 29 Coverage Remark records
2026-06-12 13:48:38 | INFO | ================================================================================
2026-06-12 13:48:38 | INFO | TABLE=mapping_storevisibility | TYPE=BRIDGE | OPERATION=INSERT
2026-06-12 13:48:38 | INFO | ================================================================================
2026-06-12 13:48:38 | INFO | Fetching Mapping Store Visibility for 2026-06-11
2026-06-12 13:48:38 | INFO | ================================================================================
2026-06-12 13:48:38 | INFO | Fetched 0 Mapping Store Visibility records
2026-06-12 13:48:38 | INFO | ================================================================================
2026-06-12 13:48:38 | INFO | TABLE=Master_VisibilityReason | TYPE=DIMENSION | OPERATION=INSERT
2026-06-12 13:48:38 | INFO | Fetching Master Visibility Reason data
2026-06-12 13:48:38 | INFO | Fetched 17 Master Visibility Reason records
2026-06-12 13:48:38 | INFO | ================================================================================
2026-06-12 13:48:38 | INFO | TABLE=Master_VisibilityDefinition | TYPE=DIMENSION | OPERATION=INSERT
2026-06-12 13:48:38 | INFO | Fetching Master Visibility Definition data
2026-06-12 13:48:38 | INFO | Fetched 861 Master Visibility Definition records
2026-06-12 13:48:38 | INFO | ================================================================================
2026-06-12 13:48:38 | INFO | TABLE=Web_Logins | TYPE=FACT | OPERATION=INSERT
2026-06-12 13:48:38 | INFO | ================================================================================
2026-06-12 13:48:38 | INFO | Fetching Web Login data for 2026-06-11
2026-06-12 13:48:38 | INFO | ================================================================================
2026-06-12 13:48:38 | INFO | Fetched 223 Web Login records
2026-06-12 13:48:38 | INFO | ================================================================================
2026-06-12 13:48:38 | INFO | TABLE=Promotion | TYPE=FACT | OPERATION=INSERT
2026-06-12 13:48:38 | INFO | ================================================================================
2026-06-12 13:48:38 | INFO | Fetching Promotion data for 818 MIDs
2026-06-12 13:48:41 | INFO | ================================================================================
2026-06-12 13:48:41 | INFO | Fetched 8,410 Promotion records
2026-06-12 13:48:41 | INFO | ================================================================================
2026-06-12 13:48:41 | INFO | TABLE=PaidVisibility | TYPE=FACT | OPERATION=INSERT
2026-06-12 13:48:41 | INFO | ================================================================================
2026-06-12 13:48:41 | INFO | Fetching Paid Visibility data for 818 MIDs
2026-06-12 13:48:48 | INFO | ================================================================================
2026-06-12 13:48:48 | INFO | Fetched 1,098 Paid Visibility records
2026-06-12 13:48:48 | INFO | ================================================================================
2026-06-12 13:48:48 | INFO | TABLE=Master_Salesterritorylayer | TYPE=DIMENSION | OPERATION=INSERT
2026-06-12 13:48:48 | INFO | Fetching Master Sales Territory Layer data
2026-06-12 13:48:48 | INFO | Fetched 33 Master Sales Territory Layer records
2026-06-12 14:57:00 | INFO | Hello from data-move Python data pipeline !
2026-06-12 14:57:00 | INFO | connecting with both db servers sql-serveras well as clickhouse DB
2026-06-12 14:57:01 | INFO | ================================================================================
2026-06-12 14:57:01 | INFO | <sqlalchemy.engine.cursor.CursorResult object at 0x0000025C294ABCD0>
2026-06-12 14:57:03 | INFO | <sqlalchemy.engine.cursor.CursorResult object at 0x0000025C2A59DBD0>
2026-06-12 14:57:04 | INFO | Both databases connected successfully
2026-06-12 14:57:04 | INFO | Collecting MIDs for: 2026-06-11
2026-06-12 14:57:04 | INFO | Found 818 MIDs
2026-06-12 14:59:11 | INFO | Hello from data-move Python data pipeline !
2026-06-12 14:59:11 | INFO | connecting with both db servers sql-serveras well as clickhouse DB
2026-06-12 14:59:12 | INFO | ================================================================================
2026-06-12 14:59:12 | INFO | <sqlalchemy.engine.cursor.CursorResult object at 0x000001423113BCD0>
2026-06-12 14:59:14 | INFO | <sqlalchemy.engine.cursor.CursorResult object at 0x00000142322F9BD0>
2026-06-12 14:59:14 | INFO | Both databases connected successfully
2026-06-12 14:59:14 | INFO | Collecting MIDs for: 2026-06-11
2026-06-12 14:59:14 | INFO | Found 818 MIDs
2026-06-12 14:59:14 | INFO | ================================================================================
2026-06-12 14:59:14 | INFO | TABLE=SOS_OneApp | TYPE=FACT | OPERATION=INSERT
2026-06-12 14:59:14 | INFO | ================================================================================
2026-06-12 14:59:14 | INFO | Fetching data for 818 MIDs
2026-06-12 14:59:15 | INFO | ================================================================================
2026-06-12 14:59:15 | INFO | Fetched 3,677 rows from SQL Server
2026-06-12 14:59:15 | INFO | ================================================================================
2026-06-12 14:59:15 | INFO | TABLE=OQaD | TYPE=FACT | OPERATION=INSERT
2026-06-12 14:59:15 | INFO | ================================================================================
2026-06-12 14:59:15 | INFO | Fetching OQaD data for 818 MIDs
2026-06-12 14:59:18 | INFO | ================================================================================
2026-06-12 14:59:18 | INFO | Fetched 464 rows
2026-06-12 14:59:18 | INFO | ================================================================================
2026-06-12 14:59:18 | INFO | TABLE=Survey | TYPE=FACT | OPERATION=INSERT
2026-06-12 14:59:18 | INFO | ================================================================================
2026-06-12 14:59:18 | INFO | Fetching Survey data for 818 MIDs
2026-06-12 14:59:19 | INFO | ================================================================================
2026-06-12 14:59:19 | INFO | Fetched 142 Survey rows
2026-06-12 14:59:19 | INFO | ================================================================================
2026-06-12 14:59:19 | INFO | TABLE=additional_visibility | TYPE=FACT | OPERATION=INSERT
2026-06-12 14:59:19 | INFO | ================================================================================
2026-06-12 14:59:19 | INFO | Fetching Additional Visibility data for 818 MIDs
2026-06-12 14:59:19 | INFO | ================================================================================
2026-06-12 14:59:19 | INFO | Fetched 1,922 Additional Visibility rows
2026-06-12 14:59:19 | INFO | ================================================================================
2026-06-12 14:59:19 | INFO | TABLE=Coverage | TYPE=FACT | OPERATION=INSERT
2026-06-12 14:59:19 | INFO | ================================================================================
2026-06-12 14:59:19 | INFO | Fetching coverage data for 818 MIDs
2026-06-12 14:59:20 | INFO | ================================================================================
2026-06-12 14:59:20 | INFO | Fetched 761 rows from SQL Server
2026-06-12 14:59:20 | INFO | ================================================================================
2026-06-12 14:59:20 | INFO | TABLE=Login | TYPE=FACT | OPERATION=INSERT
2026-06-12 14:59:20 | INFO | ================================================================================
2026-06-12 14:59:20 | INFO | Fetching Login data for yesterday
2026-06-12 14:59:21 | INFO | ================================================================================
2026-06-12 14:59:21 | INFO | Fetched 479 Login rows
2026-06-12 14:59:21 | INFO | ================================================================================
2026-06-12 14:59:21 | INFO | TABLE=Stock_Details | TYPE=FACT | OPERATION=INSERT
2026-06-12 14:59:21 | INFO | ================================================================================
2026-06-12 14:59:21 | INFO | Fetching Stock Details data for 818 MIDs
2026-06-12 14:59:26 | INFO | Fetched 41,628 Stock Details rows
2026-06-12 14:59:26 | INFO | ================================================================================
2026-06-12 14:59:26 | INFO | TABLE=Attendance | TYPE=FACT | OPERATION=DELETE+INSERT
2026-06-12 14:59:26 | INFO | Fetching Attendance data from 2026-05-27 to 2026-06-11
2026-06-12 14:59:27 | INFO | Fetched 11,918 attendance rows for 592 employees
2026-06-12 14:59:27 | INFO | ================================================================================
2026-06-12 14:59:27 | INFO | TABLE=Store_Master | TYPE=DIMENSION | OPERATION=DELETE+INSERT
2026-06-12 14:59:27 | INFO | Fetching Store Master data
2026-06-12 14:59:28 | INFO | Fetched 5,984 stores
2026-06-12 14:59:29 | INFO | ================================================================================
2026-06-12 14:59:29 | INFO | TABLE=SKU_Master | TYPE=DIMENSION | OPERATION=DELETE+INSERT
2026-06-12 14:59:29 | INFO | Fetching SKU Master data
2026-06-12 14:59:29 | INFO | Fetched 160 SKU Master rows
2026-06-12 14:59:29 | INFO | ================================================================================
2026-06-12 14:59:29 | INFO | TABLE=display_master | TYPE=DIMENSION | OPERATION=DELETE+INSERT
2026-06-12 14:59:29 | INFO | Fetching Display Master data
2026-06-12 14:59:29 | INFO | Fetched 135 Display Master records
2026-06-12 14:59:29 | INFO | ================================================================================
2026-06-12 14:59:29 | INFO | TABLE=Employee_Master | TYPE=DIMENSION | OPERATION=DELETE+INSERT
2026-06-12 14:59:29 | INFO | Fetching Employee Master data
2026-06-12 14:59:29 | INFO | Fetched 2,269 Employee Master records
2026-06-12 14:59:29 | INFO | ================================================================================
2026-06-12 14:59:29 | INFO | TABLE=Journey_Plan | TYPE=FACT | OPERATION=INSERT
2026-06-12 14:59:29 | INFO | Fetching Journey Plan for 2026-06
2026-06-12 14:59:30 | INFO | Fetched 20,976 Journey Plan records
2026-06-12 14:59:30 | INFO | ================================================================================
2026-06-12 14:59:30 | INFO | TABLE=coverage_remarks | TYPE=FACT | OPERATION=INSERT
2026-06-12 14:59:30 | INFO | Fetching Coverage Remarks
2026-06-12 14:59:30 | INFO | Fetched 29 Coverage Remark records
2026-06-12 14:59:30 | INFO | ================================================================================
2026-06-12 14:59:30 | INFO | TABLE=mapping_storevisibility | TYPE=BRIDGE | OPERATION=INSERT
2026-06-12 14:59:30 | INFO | ================================================================================
2026-06-12 14:59:30 | INFO | Fetching Mapping Store Visibility for 2026-06-11
2026-06-12 14:59:30 | INFO | Fetched 0 Mapping Store Visibility records
2026-06-12 14:59:30 | INFO | ================================================================================
2026-06-12 14:59:30 | INFO | TABLE=Master_VisibilityReason | TYPE=DIMENSION | OPERATION=INSERT
2026-06-12 14:59:30 | INFO | Fetching Master Visibility Reason data
2026-06-12 14:59:30 | INFO | Fetched 17 Master Visibility Reason records
2026-06-12 14:59:30 | INFO | ================================================================================
2026-06-12 14:59:30 | INFO | TABLE=Master_VisibilityDefinition | TYPE=DIMENSION | OPERATION=INSERT
2026-06-12 14:59:30 | INFO | Fetching Master Visibility Definition data
2026-06-12 14:59:30 | INFO | Fetched 861 Master Visibility Definition records
2026-06-12 14:59:30 | INFO | ================================================================================
2026-06-12 14:59:30 | INFO | TABLE=Web_Logins | TYPE=FACT | OPERATION=INSERT
2026-06-12 14:59:30 | INFO | ================================================================================
2026-06-12 14:59:30 | INFO | Fetching Web Login data for 2026-06-11
2026-06-12 14:59:31 | INFO | Fetched 223 Web Login records
2026-06-12 14:59:31 | INFO | ================================================================================
2026-06-12 14:59:31 | INFO | TABLE=Promotion | TYPE=FACT | OPERATION=INSERT
2026-06-12 14:59:31 | INFO | Fetching Promotion data for 818 MIDs
2026-06-12 14:59:33 | INFO | Fetched 8,410 Promotion records
2026-06-12 14:59:33 | INFO | ================================================================================
2026-06-12 14:59:33 | INFO | TABLE=PaidVisibility | TYPE=FACT | OPERATION=INSERT
2026-06-12 14:59:33 | INFO | Fetching Paid Visibility data for 818 MIDs
2026-06-12 14:59:41 | INFO | Fetched 1,098 Paid Visibility records
2026-06-12 14:59:41 | INFO | ================================================================================
2026-06-12 14:59:41 | INFO | TABLE=Master_Salesterritorylayer | TYPE=DIMENSION | OPERATION=INSERT
2026-06-12 14:59:41 | INFO | Fetching Master Sales Territory Layer data
2026-06-12 14:59:41 | INFO | Fetched 33 Master Sales Territory Layer records
2026-06-12 15:20:11 | INFO | ================================================================================
2026-06-12 15:20:11 | INFO | Hello from data-move Python data pipeline !
2026-06-12 15:20:11 | INFO | Data-pipeline running Date is -:2026-06-11
2026-06-12 15:20:11 | INFO | connecting with both db servers sql-serveras well as clickhouse DB
2026-06-12 15:20:12 | INFO | <sqlalchemy.engine.cursor.CursorResult object at 0x000002861823BCD0>
2026-06-12 15:20:13 | INFO | <sqlalchemy.engine.cursor.CursorResult object at 0x000002861939DBD0>
2026-06-12 15:20:14 | INFO | Both databases connected successfully
2026-06-12 15:20:14 | INFO | Collecting MIDs for: 2026-06-11
2026-06-12 15:20:14 | INFO | Found 818 MIDs
2026-06-12 15:20:14 | INFO | ================================================================================
2026-06-12 15:20:14 | INFO | TABLE=SOS_OneApp | TYPE=FACT | OPERATION=INSERT
2026-06-12 15:20:14 | INFO | Fetching data for 818 MIDs
2026-06-12 15:20:16 | INFO | Fetched 3,677 rows from SQL Server
2026-06-12 15:20:16 | INFO | ================================================================================
2026-06-12 15:20:16 | INFO | TABLE=OQaD | TYPE=FACT | OPERATION=INSERT
2026-06-12 15:20:16 | INFO | Fetching OQaD data for 818 MIDs
2026-06-12 15:20:20 | INFO | Fetched 464 rows
2026-06-12 15:20:20 | INFO | ================================================================================
2026-06-12 15:20:20 | INFO | TABLE=Survey | TYPE=FACT | OPERATION=INSERT
2026-06-12 15:20:20 | INFO | Fetching Survey data for 818 MIDs
2026-06-12 15:20:21 | INFO | Fetched 142 Survey rows
2026-06-12 15:20:21 | INFO | ================================================================================
2026-06-12 15:20:21 | INFO | TABLE=additional_visibility | TYPE=FACT | OPERATION=INSERT
2026-06-12 15:20:21 | INFO | Fetching Additional Visibility data for 818 MIDs
2026-06-12 15:20:22 | INFO | Fetched 1,922 Additional Visibility rows
2026-06-12 15:20:22 | INFO | ================================================================================
2026-06-12 15:20:22 | INFO | TABLE=Coverage | TYPE=FACT | OPERATION=INSERT
2026-06-12 15:20:22 | INFO | Fetching coverage data for 818 MIDs
2026-06-12 15:20:24 | INFO | Fetched 761 rows from SQL Server
2026-06-12 15:20:24 | INFO | ================================================================================
2026-06-12 15:20:24 | INFO | TABLE=Login | TYPE=FACT | OPERATION=INSERT
2026-06-12 15:20:24 | INFO | Fetching Login data for yesterday
2026-06-12 15:20:25 | INFO | Fetched 479 Login rows
2026-06-12 15:20:25 | INFO | ================================================================================
2026-06-12 15:20:25 | INFO | TABLE=Stock_Details | TYPE=FACT | OPERATION=INSERT
2026-06-12 15:20:25 | INFO | Fetching Stock Details data for 818 MIDs
2026-06-12 15:20:30 | INFO | Fetched 41,628 Stock Details rows
2026-06-12 15:20:30 | INFO | ================================================================================
2026-06-12 15:20:30 | INFO | TABLE=Attendance | TYPE=FACT | OPERATION=DELETE+INSERT
2026-06-12 15:20:30 | INFO | Fetching Attendance data from 2026-05-27 to 2026-06-11
2026-06-12 15:20:31 | INFO | Fetched 11,918 attendance rows for 592 employees
2026-06-12 15:20:31 | INFO | ================================================================================
2026-06-12 15:20:31 | INFO | TABLE=Store_Master | TYPE=DIMENSION | OPERATION=DELETE+INSERT
2026-06-12 15:20:31 | INFO | Fetching Store Master data
2026-06-12 15:20:33 | INFO | Fetched 5,984 stores
2026-06-12 15:20:33 | INFO | ================================================================================
2026-06-12 15:20:33 | INFO | TABLE=SKU_Master | TYPE=DIMENSION | OPERATION=DELETE+INSERT
2026-06-12 15:20:33 | INFO | Fetching SKU Master data
2026-06-12 15:20:33 | INFO | Fetched 160 SKU Master rows
2026-06-12 15:20:33 | INFO | ================================================================================
2026-06-12 15:20:33 | INFO | TABLE=display_master | TYPE=DIMENSION | OPERATION=DELETE+INSERT
2026-06-12 15:20:33 | INFO | Fetching Display Master data
2026-06-12 15:20:33 | INFO | Fetched 135 Display Master records
2026-06-12 15:20:33 | INFO | ================================================================================
2026-06-12 15:20:33 | INFO | TABLE=Employee_Master | TYPE=DIMENSION | OPERATION=DELETE+INSERT
2026-06-12 15:20:33 | INFO | Fetching Employee Master data
2026-06-12 15:20:34 | INFO | Fetched 2,269 Employee Master records
2026-06-12 15:20:34 | INFO | ================================================================================
2026-06-12 15:20:34 | INFO | TABLE=Journey_Plan | TYPE=FACT | OPERATION=INSERT
2026-06-12 15:20:34 | INFO | Fetching Journey Plan for 2026-06
2026-06-12 15:20:35 | INFO | Fetched 20,979 Journey Plan records
2026-06-12 15:20:35 | INFO | ================================================================================
2026-06-12 15:20:35 | INFO | TABLE=coverage_remarks | TYPE=FACT | OPERATION=INSERT
2026-06-12 15:20:35 | INFO | Fetching Coverage Remarks
2026-06-12 15:20:35 | INFO | Fetched 29 Coverage Remark records
2026-06-12 15:20:35 | INFO | ================================================================================
2026-06-12 15:20:35 | INFO | TABLE=mapping_storevisibility | TYPE=BRIDGE | OPERATION=INSERT
2026-06-12 15:20:35 | INFO | Fetching Mapping Store Visibility for 2026-06-11
2026-06-12 15:20:35 | INFO | Fetched 0 Mapping Store Visibility records
2026-06-12 15:20:35 | INFO | ================================================================================
2026-06-12 15:20:35 | INFO | TABLE=Master_VisibilityReason | TYPE=DIMENSION | OPERATION=INSERT
2026-06-12 15:20:35 | INFO | Fetching Master Visibility Reason data
2026-06-12 15:20:35 | INFO | Fetched 17 Master Visibility Reason records
2026-06-12 15:20:35 | INFO | ================================================================================
2026-06-12 15:20:35 | INFO | TABLE=Master_VisibilityDefinition | TYPE=DIMENSION | OPERATION=INSERT
2026-06-12 15:20:35 | INFO | Fetching Master Visibility Definition data
2026-06-12 15:20:35 | INFO | Fetched 861 Master Visibility Definition records
2026-06-12 15:20:35 | INFO | ================================================================================
2026-06-12 15:20:35 | INFO | TABLE=Web_Logins | TYPE=FACT | OPERATION=INSERT
2026-06-12 15:20:35 | INFO | Fetching Web Login data for 2026-06-11
2026-06-12 15:20:36 | INFO | Fetched 223 Web Login records
2026-06-12 15:20:36 | INFO | ================================================================================
2026-06-12 15:20:36 | INFO | TABLE=Promotion | TYPE=FACT | OPERATION=INSERT
2026-06-12 15:20:36 | INFO | Fetching Promotion data for 818 MIDs
2026-06-12 15:20:39 | INFO | Fetched 8,410 Promotion records
2026-06-12 15:20:39 | INFO | ================================================================================
2026-06-12 15:20:39 | INFO | TABLE=PaidVisibility | TYPE=FACT | OPERATION=INSERT
2026-06-12 15:20:39 | INFO | Fetching Paid Visibility data for 818 MIDs
2026-06-12 15:20:47 | INFO | Fetched 1,098 Paid Visibility records
2026-06-12 15:20:47 | INFO | ================================================================================
2026-06-12 15:20:47 | INFO | TABLE=Master_Salesterritorylayer | TYPE=DIMENSION | OPERATION=INSERT
2026-06-12 15:20:47 | INFO | Fetching Master Sales Territory Layer data
2026-06-12 15:20:47 | INFO | Fetched 33 Master Sales Territory Layer records