2. Problem solving metrics

!pip install amlgym

Learn a model to be evaluated:

from amlgym.benchmarks import get_domain_path, get_trajectories_path
from amlgym.algorithms import get_algorithm
from amlgym.util.util import empty_domain

# Get learning algorithm
learner = get_algorithm('offlam')

# Get an input domain with no preconditions and effects
domain_ref_path = get_domain_path('blocksworld')
domain_empty_path = empty_domain(domain_ref_path)

# Get a set of trajectories to learn from
traj_paths = get_trajectories_path('blocksworld')

# Learn a domain model
model = learner.learn(domain_empty_path, traj_paths)

Print the learned model

print(model)

Save the learned model to a file:

domain_eval_path = 'domain_learned.pddl'
with open(domain_eval_path, 'w') as f:
    f.write(model)

Get the set of problems necessary to evaluate problem solving metrics:

from amlgym.benchmarks import get_problems_path
from pprint import pprint

probs_paths = get_problems_path('blocksworld', kind='solving')
pprint(probs_paths)

Solve the problems with the model to be evaluated and evaluate it in a simulator defined by a reference model:

from amlgym.metrics import problem_solving
import unified_planning
unified_planning.shortcuts.get_environment().credits_stream = None

metrics = problem_solving(domain_eval_path, domain_ref_path, probs_paths, timeout=60)
print(metrics)