Senzing v3 G2Engine Redo Processing

Senzing v4 is released! Unless you are working with an existing Senzing v3 deployment, check out the v4 Python SDK!

Redo processing

Redo records are automatically created when certain conditions occur during entity resolution where additional processing is required. The most common reasons are:

  • A feature value becomes generic and previous decisions need to be revisited
  • Clean up after some record deletes
  • Detected related entities were being changed at the same time
  • A table inconsistency exists, potentially after a non-graceful shutdown

countRedoRecords

countRedoRecords() returns the integer count of the remaining internally queued redo records in the Senzing repository.

g2_engine.countRedoRecords()
Click to expand `countRedoRecords()` example Click to collapse
Example
#! /usr/bin/env python3

from senzing import G2Engine, G2Exception

# REPLACE /home/user/your_project with the path to your Senzing project
senzing_engine_configuration_json = '{ "PIPELINE": { "CONFIGPATH": "/home/user/your_project/etc", "SUPPORTPATH": "/home/user/your_project/data", "RESOURCEPATH": "/home/user/your_project/resources" }, "SQL": { "CONNECTION": "sqlite3://na:na@/home/user/your_project/var/sqlite/G2C.db" } }'

g2_engine = G2Engine()

try:
    g2_engine.init("G2Engine", senzing_engine_configuration_json)

    result_count = g2_engine.countRedoRecords()

    g2_engine.destroy()

    print(result_count)

except G2Exception as err:
    print(g2_engine.getLastException())

Response message:

5

getRedoRecord

getRedoRecord() retrieves the next internally queued redo record into the Senzing repository.

Unlike processRedoRecord(), getRedoRecord() does not actually process the record. To process the record, use process() 

g2_engine.getRedoRecord(response_bytearray)
Parameters
  • response_bytearray: (bytearray) Object to store the output of the method.
Click to expand `getRedoRecord()` example Click to collapse
Example
#! /usr/bin/env python3

from senzing import G2Engine, G2Exception

# REPLACE /home/user/your_project with the path to your Senzing project
senzing_engine_configuration_json = '{ "PIPELINE": { "CONFIGPATH": "/home/user/your_project/etc", "SUPPORTPATH": "/home/user/your_project/data", "RESOURCEPATH": "/home/user/your_project/resources" }, "SQL": { "CONNECTION": "sqlite3://na:na@/home/user/your_project/var/sqlite/G2C.db" } }'

g2_engine = G2Engine()

response_bytearray = bytearray()

try:
    g2_engine.init("G2Engine", senzing_engine_configuration_json)

    g2_engine.getRedoRecord(response_bytearray)

    print(response_bytearray.decode())

except G2Exception as err:
    print(err)
Output
{
  "REASON": "LIB_FEAT_ID[13] of FTYPE_ID[7] went generic for CANDIDATES ECLASS_ID[1] in LENS_ID[1]",
  "DATA_SOURCE": "TEST",
  "RECORD_ID": "39F84F26E1F85ABCCE20D5FDBCA7D08729DD829A",
  "ENTITY_TYPE": "GENERIC",
  "DSRC_ACTION": "X"
}

process

process() processes the redo record retrieved using getRedoRecord().

g2_engine.process(response_bytearray)
Parameters
  • response_bytearray: (bytearray) the redo record obtained by getRedoRecord() to process.
Click to expand `process()` example Click to collapse 
#! /usr/bin/env python3

from senzing import G2Engine, G2Exception

# REPLACE /home/user/your_project with the path to your Senzing project
senzing_engine_configuration_json = '{ "PIPELINE": { "CONFIGPATH": "/home/user/your_project/etc", "SUPPORTPATH": "/home/user/your_project/data", "RESOURCEPATH": "/home/user/your_project/resources" }, "SQL": { "CONNECTION": "sqlite3://na:na@/home/user/your_project/var/sqlite/G2C.db" } }'

g2_engine = G2Engine()

response_bytearray = bytearray()

try:
    g2_engine.init("G2Engine", senzing_engine_configuration_json)

    g2_engine.getRedoRecord(response_bytearray)

    g2_engine.process(response_bytearray)

    print(response_bytearray.decode())

except G2Exception as err:
    print(err)

processWithInfo

processWithInfo() processes the redo record retrieved using getRedoRecord(), and returns a JSON document containing the ENTITY_ID values of the affected entities.

g2_engine.processWithInfo(response_bytearray, process_response_bytearray)
Parameters
  • response_bytearray: (bytearray) the redo record obtained by getRedoRecord() to process.
  • process_response_bytearray: (bytearray) Object to store the output of the method.
Click to expand `processWithInfo()` example Click to collapse
Example
#! /usr/bin/env python3

from senzing import G2Engine, G2Exception

# REPLACE /home/user/your_project with the path to your Senzing project
senzing_engine_configuration_json = '{ "PIPELINE": { "CONFIGPATH": "/home/user/your_project/etc", "SUPPORTPATH": "/home/user/your_project/data", "RESOURCEPATH": "/home/user/your_project/resources" }, "SQL": { "CONNECTION": "sqlite3://na:na@/home/user/your_project/var/sqlite/G2C.db" } }'

g2_engine = G2Engine()

response_bytearray = bytearray()

with_info_bytearray = bytearray()

try:
    g2_engine.init("G2Engine", senzing_engine_configuration_json)

    g2_engine.getRedoRecord(response_bytearray)

    g2_engine.processWithInfo(
        response_bytearray,
        with_info_bytearray)

    print(with_info_bytearray.decode())

except G2Exception as err:
    print(err)
Output
{
  "REASON": "LIB_FEAT_ID[1307] of FTYPE_ID[14] went generic for CANDIDATES AND SCORING ECLASS_ID[1] in LENS_ID[1]",
  "DATA_SOURCE": "CUSTOMERS",
  "RECORD_ID": "2181",
  "ENTITY_TYPE": "GENERIC",
  "DSRC_ACTION": "X"
}

processRedoRecord

processRedoRecordWithInfo() processes the next redo record in the redo queue. If processRedoRecord() returns a blank response_bytearray, then the redo queue is empty at the point of execution. Additional redo records can be created after reevaluation.

g2_engine.processRedoRecord(response_bytearray)
Parameters
  • response_bytearray: (bytearray) Object to store the output of the method.
Click to expand `processRedoRecord()` example Click to collapse
Example
#! /usr/bin/env python3

from senzing import G2Engine, G2Exception

# REPLACE /home/user/your_project with the path to your Senzing project
senzing_engine_configuration_json = '{ "PIPELINE": { "CONFIGPATH": "/home/user/your_project/etc", "SUPPORTPATH": "/home/user/your_project/data", "RESOURCEPATH": "/home/user/your_project/resources" }, "SQL": { "CONNECTION": "sqlite3://na:na@/home/user/your_project/var/sqlite/G2C.db" } }'

g2_engine = G2Engine()

try:
    g2_engine.init("G2Engine", senzing_engine_configuration_json)

    g2_engine.processRedoRecord()

except G2Exception as err:
    print(err)
Output
{
  "DATA_SOURCE": "CUSTOMERS",
  "RECORD_ID": "2192",
  "AFFECTED_ENTITIES": [
    {
      "ENTITY_ID": 105
    }
  ],
  "INTERESTING_ENTITIES": {
    "ENTITIES": []
  }
}

processRedoRecordWithInfo

processRedoRecordWithInfo() processes the next redo record in the redo queue, and returns a JSON document containing the ENTITY_ID values of the affected entities.

g2_engine.processRedoRecordWithInfo(response_bytearray, with_info_bytearray)
Parameters
  • response_bytearray: (bytearray) Object to store the output of the method.
  • info_bytearray: (bytearray) Returns the “withInfo” section
Click to expand `processRedoRecordWithInfo()` example Click to collapse
Example
#! /usr/bin/env python3

from senzing import G2Engine, G2Exception

# REPLACE /home/user/your_project with the path to your Senzing project
senzing_engine_configuration_json = '{ "PIPELINE": { "CONFIGPATH": "/home/user/your_project/etc", "SUPPORTPATH": "/home/user/your_project/data", "RESOURCEPATH": "/home/user/your_project/resources" }, "SQL": { "CONNECTION": "sqlite3://na:na@/home/user/your_project/var/sqlite/G2C.db" } }'

g2_engine = G2Engine()

response_bytearray = bytearray()

with_info_bytearray = bytearray()

try:
    g2_engine.init("G2Engine", senzing_engine_configuration_json)

    g2_engine.processRedoRecordWithInfo(
        response_bytearray,
        with_info_bytearray)

    print(with_info_bytearray.decode())

except G2Exception as err:
    print(err)
Output
{
  "DATA_SOURCE": "CUSTOMERS",
  "RECORD_ID": "2192",
  "AFFECTED_ENTITIES": [
    {
      "ENTITY_ID": 105
    }
  ],
  "INTERESTING_ENTITIES": {
    "ENTITIES": []
  }
}

If you have any questions, contact Senzing Support. Support is 100% FREE!