3. Predictive power metrics

!pip install amlgym

Learn a model to be evaluated:

from amlgym.benchmarks import get_domain_path, get_trajectories_path
from amlgym.algorithms import get_algorithm
from amlgym.util.util import empty_domain

# Get learning algorithm
learner = get_algorithm('nolam')

# Get an input domain with no preconditions and effects
domain_ref_path = get_domain_path('blocksworld')
domain_empty_path = empty_domain(domain_ref_path)

# Get a set of trajectories to learn from
traj_paths = get_trajectories_path('blocksworld')

# Learn a domain model
model = learner.learn(domain_empty_path, traj_paths)

Print the learned model

print(model)

Save the learned model to a file:

domain_eval_path = 'domain_learned.pddl'
with open(domain_eval_path, 'w') as f:
    f.write(model)

Get the test set of states necessary to evaluate predicted applicability and predicted effects metrics:

from amlgym.benchmarks import get_problems_path, get_test_states

test_states = get_test_states('blocksworld', kind='predictive_power')
problem_paths = get_problems_path('blocksworld', kind='predictive_power')

# get first problem file path
problem_path = problem_paths[0]

# get test set of state for a single problem
test_states = test_states[problem_path.split('/')[-1]]

Create a simulator defined by the learned domain model and an environment simulator defined by a reference domain model

from amlgym.modeling.UPEnv import UPEnv
simulator_learned = UPEnv(domain_eval_path, problem_path)
simulator_ref = UPEnv(domain_ref_path, problem_path)

Evaluate the predicted applicability and predicted effects metrics with a learned domain simulator and environment simulator.

from amlgym.metrics import predictive_power
predictive_metrics = predictive_power(simulator_learned,
                                      simulator_ref,
                                      test_states)
print(predictive_metrics)