Tutorial AAAI 2026

AMLGym provides several benchmark domains from previous Internaional Planning Competitions (IPCs).

!pip install amlgym > /dev/null 2>&1

from amlgym.benchmarks import print_domains

print_domains()

The PDDL model of each domain can be inspected:

from amlgym.benchmarks import get_domain_path

domain_path = get_domain_path('blocksworld')
print(domain_path)

from amlgym.benchmarks import get_domain

domain_pddl = get_domain('blocksworld')
print(domain_pddl)

A set of 10 trajectories is provided for each domain:

from amlgym.benchmarks import get_trajectories_path
from pprint import pprint

trajectory_paths = get_trajectories_path('blocksworld')
pprint(trajectory_paths)

An example of trajectory for the blocksworld domain is:

from amlgym.benchmarks import get_trajectories

trajectory_idx = 0
trajectory = get_trajectories('blocksworld')[trajectory_idx]
print(trajectory)

The problem used for generating the trajectory can be obtained by:

from amlgym.benchmarks import get_problems_path

problem_path = get_problems_path('blocksworld')[trajectory_idx]
print(problem_path)

with open(problem_path, 'r') as f:
    print(f.read())

Passive Algorithms

AMLGym includes several algorithms for passive learning domain models from full/partial/noisy trajectories.

from amlgym.algorithms import print_algorithms

print_algorithms()

Passive learning with full observability

Instantiate an algorithm for passive learning domain models in fully observable environments (e.g., SAM):

from amlgym.algorithms import get_algorithm

learner = get_algorithm('sam')

Get an empty IPC domain:

from amlgym.util.util import empty_domain

domain_path = get_domain_path('blocksworld')

domain_empty_path = empty_domain(domain_path)
with open(domain_empty_path, 'r') as f:
    print(f.read())

Get a set of trajectories associated with the domain:

traj_paths = get_trajectories_path('blocksworld')

Learn a domain model specified in PDDL:

model = learner.learn(domain_empty_path, traj_paths)
print(model)

Save the learned model to a file

domain_learned_path = 'domain_learned.pddl'
with open(domain_learned_path, 'w') as f:
    f.write(model)

Evaluating a domain model

AMLGym provides several measures for evaluation an action model w.r.t. a reference model or simulator.

Syntactic Similarity

Syntactic similarity can be measured by means of precision and recall:

from amlgym.metrics import syntactic_precision
from pprint import pprint

domain_ref_path = get_domain_path('blocksworld')
precision = syntactic_precision(domain_learned_path, domain_ref_path)
pprint(precision)

from amlgym.metrics import syntactic_recall

recall = syntactic_recall(domain_learned_path, domain_ref_path)
pprint(recall)

Problem solving

Get the set of problems necessary to evaluate problem solving metrics:

probs_paths = get_problems_path('blocksworld', kind='solving')
pprint(probs_paths)

Solve the problems with the model to be evaluated and evaluate it in a simulator defined by a reference model:

from amlgym.metrics import problem_solving
import unified_planning
unified_planning.shortcuts.get_environment().credits_stream = None

metrics = problem_solving(domain_learned_path, domain_ref_path, probs_paths, timeout=60)
pprint(metrics)

Predictive power

Get the test set of states necessary to evaluate predicted applicability and predicted effects metrics:

from amlgym.benchmarks import get_test_states

test_states = get_test_states('blocksworld')
print(len(test_states.keys()))
pprint(next(iter(test_states.keys())))

problem_paths = get_problems_path('blocksworld', kind='predictive_power')

# get first problem file path
problem_path = problem_paths[0]

# get test set of state for a single problem
test_states = test_states[problem_path.split('/')[-1]]
pprint(test_states)

Create a simulator defined by the learned domain model and an environment simulator defined by a reference domain model

from amlgym.modeling.UPEnv import UPEnv

simulator_learned = UPEnv(domain_learned_path, problem_path)
simulator_ref = UPEnv(domain_ref_path, problem_path)

Evaluate the predicted applicability and predicted effects metrics with a learned domain simulator and environment simulator.

from amlgym.metrics import predictive_power

predictive_metrics = predictive_power(simulator_learned,
                                      simulator_ref,
                                      test_states)
pprint(predictive_metrics)

Comparing passive learning performances with full observability

Question A: Who achieves higher syntactic precision between NOLAM and OffLAM in domain tpp?

Instantiate the learning algorithms:

offlam = get_algorithm('offlam')
nolam = get_algorithm('nolam', noise=0.)

Get an input (empty IPC) domain for tpp and the associated set of learning trajectories:

domain_path = get_domain_path('tpp')
domain_empty_path = empty_domain(domain_path)
traj_paths = get_trajectories_path('tpp')

Learn a domain model specified in PDDL:

# Learn a domain model using OffLAM
domain_offlam = offlam.learn(domain_empty_path, traj_paths)

# Learn a domain model using NOLAM
domain_nolam = nolam.learn(domain_empty_path, traj_paths)

Store the learned models for evaluation

domain_offlam_path = 'offlam.pddl'
with open(domain_offlam_path, 'w') as f:
    f.write(domain_offlam)

domain_nolam_path = 'nolam.pddl'
with open(domain_nolam_path, 'w') as f:
    f.write(domain_nolam)

# ...

Question B: Who achieves the highest solving ratio among SAM and OffLAM in domain goldminer?

# ...