Complexa Evaluate Pdbs

Evaluate-from-PDB-Dir Config Reference

Companion to SKILL.md. Every evaluate_*_from_pdb_dir.yaml and its paired analyze_*.yaml, with the result_type they emit, the dataset.* keys they require, and the metric.* / aggregation.* knobs they accept. Three worked examples at the bottom.

1. Design type → config matrix

Notes:

• The protein-binder and ligand-binder cases share evaluate_from_pdb_dir.yaml; switch behavior by setting result_type, metric.binder_folding_method, and metric.inverse_folding_model on the CLI.

• There is no shipped evaluate_motif_protein_binder_from_pdb_dir.yaml; reuse configs/evaluate_motif_binder.yaml with ++input_mode=pdb_dir ++sample_storage_path=<dir> and an appropriate dataset/motif_target_dict_cfg override.

2. Evaluate config schema

All evaluate_* configs share a top-level shape (run identification, input_mode, protein_type, sample_storage_path, output_dir, eval_njobs, seed, ncpus_, dataset.*, metric.*). Below: the differences that matter when running from a PDB directory.

evaluate_from_pdb_dir.yaml

• protein_type: binder (one config handles both protein and ligand binders).

• input_mode: pdb_dir (already set; never override back to generated).

• defaults: - generation/targets_dict@dataset — resolves dataset.task_name against configs/targets/targets_dict.yaml (and ligand_targets_dict via task naming).

• Required dataset.*:

  • dataset.task_name — target key (e.g. 02_PDL1, 39_7V11_LIGAND).

• Key metric.* fields:

  • compute_binder_metrics: true.

  • binder_folding_method — picks the refolding backend (table above).

  • sequence_types: [self|mpnn|mpnn_fixed] — which sequence(s) to refold.

  • num_redesign_seqs — MPNN sequence count (default 8 here).

  • interface_cutoff — Å cutoff for interface residue detection (default 8.0 protein, 6.0 motif).

  • inverse_folding_model — soluble_mpnn / protein_mpnn (protein) or ligand_mpnn (ligand).

  • compute_pre_refolding_metrics, pre_refolding.{bioinformatics,tmol,hbplus} — optional pre-refold interface metrics.

  • compute_refolded_structure_metrics, refolded.{bioinformatics,tmol,hbplus} — optional post-refold.

  • compute_monomer_metrics, compute_designability, compute_codesignability, designability_modes, codesignability_modes — monomer-on-binder eval.

  • compute_co_sequence_recovery, compute_ss — sequence/secondary-structure metrics.

  • compute_novelty_{pdb,afdb,afdb_rep_v4} — FoldSeek novelty against known DBs.

  • keep_folding_outputs — keep refolded PDBs.

• File walk control:

  • ignore_generated_pdb_suffix: "_binder.pdb" (default) — drop intermediate binder-only PDBs from the walk.

  • file_limit: N — cap input count.

• result_type is set inline (ligand_binder or protein_binder) and propagates to the paired analyze step.

evaluate_ame_from_pdb_dir.yaml

• protein_type: motif_binder.

• defaults: - /design_tasks/ame_dict_v2@dataset — resolves AME tasks (M0024_1nzy, M0096_1chm, etc.).

• Required dataset.task_name — must match a key in ame_dict_v2.

• Key metric.*:

  • compute_motif_binder_metrics: True.

  • binder_folding_method: rf3_latest (only RF3 makes sense for ligand motif binders).

  • inverse_folding_model: ligand_mpnn.

  • sequence_types: [mpnn_fixed, self] (default — mpnn_fixed keeps the motif residues constant).

  • interface_cutoff: 6.0.

  • compute_binder_metrics, compute_monomer_metrics — optional add-ons; compute_motif_metrics is incompatible with motif_binder.

• result_type: motif_ligand_binder.

Shared reference configs (no _from_pdb_dir suffix)

These ship for completeness; the from_pdb_dir variants are derived from them with input_mode=pdb_dir baked in.

• configs/evaluate.yaml — unified binder evaluation, defaults input_mode: generated. Pass ++input_mode=pdb_dir ++sample_storage_path=<dir> to evaluate an external directory.

• configs/evaluate_motif_binder.yaml — motif binder (protein + ligand variants). Set ++result_type=motif_protein_binder (with AF2/ColabDesign + ProteinMPNN/SolubleMPNN) or rely on the default motif_ligand_binder (RF3 + LigandMPNN). Use this when you have motif-protein-binder PDBs — there is no dedicated _from_pdb_dir variant.

3. Analyze config schema

analyze.yaml

• result_type: protein_binder (default) or ligand_binder (override).

• aggregation.analysis_modes — default [binder, monomer] for binder result types.

• aggregation.success_thresholds — null (defaults) or a dict of {metric: {threshold, op, scale, column_prefix}} entries.

• Built-in defaults:

  • protein_binder: i_pAE * 31 <= 7.0, pLDDT >= 0.9, scRMSD_ca < 1.5.

  • ligand_binder: min_ipAE * 31 < 2.0, scRMSD_ca < 2.0, ligand_scRMSD_aligned_allatom < 5.0.

• Monomer-mode thresholds (when [monomer] is in analysis_modes):

  • aggregation.designability_thresholds — {mode: {model: {threshold, op}}}; default 2.0 Å.

  • aggregation.ca_codesignability_thresholds — default 2.0 Å.

  • aggregation.allatom_codesignability_thresholds — default 2.0 Å.

  • aggregation.require_all_thresholds: false — false = OR across modes/models, true = AND.

analyze_motif_binder.yaml

• result_type: motif_protein_binder (default in the YAML) or motif_ligand_binder (override on CLI).

• aggregation.analysis_modes — default [motif_binder, binder, monomer].

• aggregation.motif_binder_success_thresholds:

  • motif_protein_binder defaults — binder: i_pAE*31 <= 7.0, pLDDT >= 0.8, scRMSD_ca < 2.0; motif: motif_rmsd_pred_all < 2.0, correct_motif_sequence_all >= 1.0.

  • motif_ligand_binder defaults — binder: scRMSD_bb3 <= 2.0; motif: motif_rmsd_pred_all <= 1.5, correct_motif_sequence_all >= 1.0, has_ligand_clashes_all < 0.5.

• A sample is successful when at least one redesign index passes every binder criterion AND every motif criterion jointly. Per-redesign joint evaluation, not pooled.

• aggregation.success_thresholds (binder-only mode) and the three monomer threshold dicts work identically to analyze.yaml.

4. motif_protein_binder vs motif_ligand_binder

Both come from evaluate_motif_binder.yaml family + analyze_motif_binder.yaml. The split is target-driven:

When in doubt: if every PDB has a small-molecule ligand chain, it is motif_ligand_binder. If the second chain is another protein receptor, it is motif_protein_binder.

5. Ligand L:0 rename for AME RF3 evaluation

Lifted from README.md "Evaluating AME Designs with Ligand Targets (RF3)":

When running RF3 evaluation on AME-generated PDB files that contain a ligand (small molecule on chain A), RF3 will attempt to add missing atoms based on the ligand's CCD code. This can cause shape errors in downstream RMSD calculations and provide the incorrect structure.

Solution: rename the ligand residue to L:0 before passing to RF3. This tells RF3 to treat it as a generic ligand and skip atom completion.

from atomworks.io import load_any, to_pdb_file
atom_array = load_any("my_design.pdb")
ligand_mask = atom_array.chain_id == "A"
atom_array.res_name[ligand_mask] = "L:0"
to_pdb_file(atom_array, "my_design_rf3_ready.pdb")

Apply this transform once over the whole input directory before invoking complexa analysis configs/evaluate_ame_from_pdb_dir.yaml .... If the PDBs came out of complexa generate with protein_type=motif_binder, the rename is already done.

6. Worked examples

Example A — protein binder PDB directory, AF2 refold

User: "Re-fold these 200 PDL1 binders with AlphaFold2 and tell me what fraction passes the default AlphaProteo thresholds."

complexa analysis configs/evaluate_from_pdb_dir.yaml \
  ++sample_storage_path=/data/pdl1_designs \
  ++dataset.task_name=02_PDL1 \
  ++metric.binder_folding_method=colabdesign \
  ++metric.inverse_folding_model=soluble_mpnn \
  ++metric.sequence_types=[self,mpnn_fixed] \
  ++metric.num_redesign_seqs=8 \
  ++result_type=protein_binder \
  ++eval_njobs=2 \
  ++run_name=pdl1_pdb_dir_af2

Pass-rate filter (applied by analyze): mpnn_complex_i_pAE * 31 <= 7.0 AND mpnn_complex_pLDDT >= 0.9 AND mpnn_binder_scRMSD_ca < 1.5.

Example B — AME PDB directory, RF3 refold (motif ligand binder)

User: "Score this folder of AME 1nzy designs — joint binder + motif success please."

# First, rename ligand to L:0 in every PDB (one-time).
python scripts/rename_ligand_to_L0.py /data/ame_1nzy_designs

complexa analysis configs/evaluate_ame_from_pdb_dir.yaml \
  ++sample_storage_path=/data/ame_1nzy_designs \
  ++dataset.task_name=M0024_1nzy \
  ++metric.binder_folding_method=rf3_latest \
  ++metric.inverse_folding_model=ligand_mpnn \
  ++metric.sequence_types=[mpnn_fixed,self] \
  ++metric.num_redesign_seqs=2 \
  ++result_type=motif_ligand_binder \
  ++run_name=ame_m0024_pdb_dir

Joint success criterion (from analyze_motif_binder.yaml, motif_ligand_binder defaults): at least one redesign index satisfies binder scRMSD_bb3 <= 2.0 AND motif motif_rmsd_pred_all <= 1.5 AND correct_motif_sequence_all >= 1.0 AND has_ligand_clashes_all < 0.5.

Example C — ligand binder PDB directory, RF3 refold

User: "Score this folder of 7V11 ligand-binder designs with RF3 and LigandMPNN."

complexa analysis configs/evaluate_from_pdb_dir.yaml \
  ++sample_storage_path=/data/7v11_designs \
  ++dataset.task_name=39_7V11_LIGAND \
  ++metric.binder_folding_method=rf3_latest \
  ++metric.inverse_folding_model=ligand_mpnn \
  ++metric.sequence_types=[self,mpnn_fixed] \
  ++metric.num_redesign_seqs=8 \
  ++result_type=ligand_binder \
  ++eval_njobs=2 \
  ++run_name=7v11_pdb_dir_rf3

Pass-rate filter (applied by analyze): min_ipAE * 31 < 2.0 AND scRMSD_ca < 2.0 AND ligand_scRMSD_aligned_allatom < 5.0. Override via ++aggregation.success_thresholds.min_ipAE.threshold=… etc.

7. Pointers

• Per-metric output column names: Evaluation & Analysis Guide, "Result CSV Reference".

• Default thresholds and Python filter examples: Evaluation & Analysis Guide, "Success Criteria" and "Reading Results in Python".

• Hardware tables (VRAM per backend, wall-clock per N PDBs): references/hardware.md.