Fix dangling agent roles

Fix dangling agent roles#

The dangling AR fixer handles bibliographic resources whose pro:isDocumentContextFor values include an agent role absent from the local RDF files. It removes every author, editor, and publisher role from each affected resource. It then prepares provider CSV files that Meta can use to recreate those fields.

The operation is all or nothing. A dry run with any blocker produces a diagnostic JSON report, exits with a nonzero status, and does not produce CSV files.

Generate the plan and CSV files#

uv run python -m oc_meta.run.patches.fix_dangling_ars \
  -c config/meta_config.yaml \
  --dry-run \
  --report-file dangling_ar_plan.json \
  --csv-output-dir dangling_ar_meta_input \
  --cache-file dangling_ar_api_cache.sqlite \
  --mailto name@example.org

The dry run reads the RDF files below output_rdf_dir/rdf. It does not query a triplestore or modify RDF. API responses and 404 results are stored in the SQLite cache. Use --refresh-cache to fetch Crossref and DataCite again.

Local RDF scans use Linux worker processes. Set their number with --workers; the default is the lower value between the available CPU count and 16.

Provider selection#

The fixer uses one provider record per BR:

  1. Query Crossref with the local DOI.

  2. If Crossref has no record, query DataCite.

  3. If DataCite also has no record, block the plan.

A provider record covers authors, editors, and the publisher together. An empty field is accepted as the provider’s value for that field.

When every BR has a provider and no RDF blocker exists, the dry run writes:

dangling_ar_meta_input/
├── crossref/
│   └── input.csv
└── datacite/
    └── input.csv

Both files use Meta’s 11 canonical columns. Each row contains the BR OMID and DOI in id, the local title and type, and the three responsible-agent fields from its selected provider. The other bibliographic fields are empty. Names and provider identifiers are serialized in Meta input syntax; Curator and Creator handle deduplication, disambiguation, and RA/AR creation.

The JSON plan records hashes for the Meta configuration, the full operation set, and both CSV files. Plans generated by the earlier reconciliation-based fixer are rejected.

RDF deletion rules#

For every affected BR, execution removes all pro:isDocumentContextFor values, including references to valid, malformed, and absent ARs. Present AR entities are deleted only when every BR that references them belongs to the plan. RA entities and their identifiers are not deleted.

An absent AR receives a deletion snapshot only when its latest local snapshot is active. No snapshot is added when the latest snapshot is already invalidated or no snapshot exists.

The dry run blocks the whole plan when:

  • a target AR is also referenced by a BR outside the plan;

  • a oco:hasNext edge connects a target AR to an AR outside the deletion set;

  • neither Crossref nor DataCite returns a record;

  • the scan is interrupted.

Invalid role types, missing holders, multiple holders, malformed chains, and ambiguous agent identifiers do not block execution because the entire local role field is removed.

Execute the plan#

uv run python -m oc_meta.run.patches.fix_dangling_ars \
  -c config/meta_config.yaml \
  --execute dangling_ar_plan.json \
  --resp-agent https://orcid.org/0000-0002-8420-0696

Progress is saved after each BR. If execution is interrupted, run the same command with the same plan and progress file. Do not run Meta while the execution report has complete: false.

Once RDF has changed, reindex_required.out is created beside the plan. It remains present after a resumed execution completes.

Reindex and rebuild the roles#

After the execution report has complete: true:

  1. Reindex the data and provenance triplestores from the RDF files.

  2. Remove reindex_required.out.

  3. Create one Meta configuration for Crossref and one for DataCite. Keep the dataset paths, counters, supplier prefix, and other processing settings equal to the correction configuration.

  4. Set input_csv_dir to the corresponding provider directory.

  5. Set source to the corresponding provider and silencer to all three responsible-agent fields.

The provider-specific settings are:

input_csv_dir: /path/to/dangling_ar_meta_input/crossref
source: https://api.crossref.org/
silencer: ["author", "editor", "publisher"]

input_csv_dir: /path/to/dangling_ar_meta_input/datacite
source: https://api.datacite.org/
silencer: ["author", "editor", "publisher"]

Run Meta once per provider configuration:

uv run python -m oc_meta.run.meta_process -c crossref_meta_config.yaml
uv run python -m oc_meta.run.meta_process -c datacite_meta_config.yaml