AutoFilter

The AutoFilter removes nodes that keep causing violations from a running optimization and reports them with their reasons, so that the remaining nodes get a feasible plan. This guide covers how the filter decides, how to switch it on in Java and in the JSON input of the REST service, its properties and the settings for single nodes. It is written for developers who plan with over-constrained input and know nodes, resources and properties from Basic elements.

Before you start

• JOpt in your Java project, see Getting started, or a running JOpt.TourOptimizer endpoint for the JSON input, see TourOptimizer REST server.
• A license if your optimization has more elements than JOpt allows without one: License guide: element limits.

What it does

In dispatch practice, infeasible input is common: resources are scarce, time windows conflict, capacity is tight, and data quality is imperfect. A handful of problematic nodes can destabilize the entire optimization: the solver spends its budget exploring infeasible regions, and the resulting schedule is unusable.

The AutoFilter analyzes which nodes cause violations across many parallel solution candidates, identifies the systematic offenders, and removes them from the optimization. The remaining nodes get a feasible plan, and the excluded tasks are listed with their reasons.

The multi-solution penalty system

The AutoFilter rests on a statistical analysis across many solution candidates. During the optimization run, JOpt creates a large number of parallel solutions. Each solution is a complete schedule of all nodes and all resources, and the AutoFilter analyzes each of them for violations.

The system works with two counters per node:

• Penalty points. Each time a node causes a violation in a solution candidate, it receives penalty points. The severity depends on the violation type and the configured margins.
• Anti-penalty (non-violation occurrences). Each time a node appears in a solution without causing a violation, this is recorded as well. These occurrences act as evidence that the node is generally feasible.

The ratio between penalty points and non-violation occurrences decides whether a node is a systematic problem or was unlucky in a few solution candidates. If the ratio becomes too unfavorable, the node is filtered out.

Figure: Two solutions for the same route. Node D is late in solution one, node A is late in solution two. Both collect penalty points. Across many such solutions, the AutoFilter finds the node that is the systematic offender.

The multi-solution approach matters because infeasibility often moves around. In one solution, node D causes a late violation. In another permutation, node A does. A single-solution analysis would blame whichever node happens to be the violator in the current schedule. The multi-solution analysis finds the nodes that are structurally problematic regardless of the route arrangement.

How it differs from penalty costs

To our best knowledge, many optimization systems manage infeasibility by raising the penalty weights for violations. This has three problems:

1. Oscillation. If the instance is infeasible, the algorithm keeps shifting violations from one node to another without converging. The penalty function says "stop being late here", but the solver can only comply by being late somewhere else.
2. Contamination. A small set of truly infeasible outliers prevents a good plan for the feasible majority. The solver spends its optimization budget on nodes that can never be satisfied, and the quality for all other nodes degrades.
3. Fragility. Penalty weights must be tuned per scenario. A weight that works for 200 nodes in Berlin may fail for 300 nodes in Munich. This makes a deployment brittle and hard to maintain.

The AutoFilter removes the node from the problem instead of raising its cost. The solver then optimizes the remaining nodes and produces a feasible plan. The removed nodes are reported with their filter reasons, so the business can act on them: relax time windows, add resources, or move the task to the next day.

To our best knowledge, none of the other routing engines we checked (Google OR-Tools, Timefold, jsprit, VROOM) documents a filter that collects violation statistics per node across many solution candidates and removes the systematic offenders during the run. To our best knowledge, these engines leave jobs unassigned when not all can be served, and a drop penalty or a priority in the objective decides which.

How the filter decides

Besides the ratio of the two counters, the AutoFilter applies these rules:

Route reordering before filtering. Before removing a node, the AutoFilter checks whether rearranging the route can resolve the violation. If a different visit order eliminates the problem, the node stays. Filtering is a last resort, not a first response.

Priority-aware substitution. Nodes can carry importance values. When filtering is necessary, AutoFilter prefers to remove low-priority nodes over high-priority ones. If removing a low-priority node resolves the violation that a high-priority node is causing, the low-priority node is filtered instead. This means contractually important tasks are protected at the expense of optional or deferrable work.

Collective penalization with importance threshold. When an entire route exceeds its maximal working time or its maximal distance, all nodes on that route receive penalty points. However, nodes with importance above a configurable threshold (JOpt.AutoFilter.collectivePenalization.maxImportance) are exempt from this collective penalty. This prevents high-value tasks from being punished for route-level problems they did not cause.

Graceful filtering. The property JOpt.AutoFilter.tryGracefulNodeFiltering enables a check after identifying nodes for filtering: if unassigning only a subset of the candidates is sufficient to resolve the violation, the remaining candidates are kept. This minimizes the number of excluded nodes.

Cluster protection. When nodes are part of a joined-visit-duration cluster (co-located tasks), the property JOpt.AutoFilter.tryAvoidSplittingJoinedClusters can prevent the AutoFilter from breaking apart these clusters unnecessarily.

Strict mode on the last execution. The property JOpt.AutoFilter.useStrictFilterOnLastExecution switches on a strict filtering pass on the last execution of the AutoFilter. This ensures the final solution is stabilized: any nodes that are still causing violations after the full optimization run are removed in this final pass.

When to use it

The AutoFilter is for input that can be infeasible, when the business accepts a feasible plan for most nodes plus a list of excluded tasks. Step 1 of Rollout in steps gives the rule.

AutoFilter and pillar nodes

Pillars (CapturedNodes) enforce SLAs by architecture; see Pillar nodes. The AutoFilter manages infeasibility by removal. The two complement each other: use pillars for "must meet SLA" contracts, and use the AutoFilter when you accept the best possible plan under infeasibility. A pillar can be protected from AutoFilter filtering.

AutoFilter and cost weights

Cost weights (JOptWeight.*) decide the preference among feasible or near-feasible solutions. The AutoFilter changes which nodes are considered at all. Tune the weights to improve the solution quality, and switch on the AutoFilter when you must handle infeasible input.

AutoFilter and resource constraints

When resource mismatch filtering is switched on, nodes that are systematically assigned to the wrong resource (violating mandatory or preferred constraints) accumulate penalty points. This shows the tasks that the available workforce cannot serve with its skill and territory constraints.

How to use it

Selective AutoFilter in Java

The selective mode switches the AutoFilter on only for the violation categories you name. It is the recommended mode for production, because you control what can trigger filtering. Start with one category, typically late arrival, look at the results, and add categories as needed.

props.setProperty("JOpt.SelectiveAutoFilter", "TRUE");
props.setProperty("JOpt.SelectiveAutoFilter.TimeWindow.Late", "TRUE");

Full AutoFilter in Java

props.setProperty("JOpt.AutoFilter", "TRUE");

This switches the AutoFilter on for all violation categories, as a comment in AutoFilterLateExample.java notes.

Filtering a particular node

A local IAutoFilterNodeConstraint can be attached to single nodes. Only these nodes are subject to that filtering rule. This node-level approach suits production scenarios in which not all tasks can be deferred in the same way.

// Create a late filter constraint
IAutoFilterNodeConstraint lateFilter = new LateAutoFilterConstraint();

// Attach only to the deferrable node
List<IAutoFilterNodeConstraint> constraints = new ArrayList<>();
constraints.add(lateFilter);
nuernberg.setAutoFilterConstraints(constraints);

The pattern suits:

• nodes representing optional or low-priority work;
• unconfirmed appointments that may be deferred;
• tasks from external systems where data quality is uncertain.

The example AutoFilterParticularNodeExample.java attaches a late filter constraint to the node Nuernberg only.

Protecting nodes from filtering

Individual nodes can be marked as protected. A protected node will not be filtered regardless of its violation count. Use this for contractually mandatory tasks.

The node method setMinAutoFilterProtectedExecutions(int) sets the number of optimization executions during which the AutoFilter does not remove the node. After that number, the AutoFilter may exclude it. The property JOpt.AutoFilter.canDisableNodeProtection allows the AutoFilter to remove the protection once the number of protected executions is reached.

Filter events as a callback

JOpt reports filter events in two forms. The first is a callback, with two methods:

void onNodeFiltering(int code, String message, List<INodeFilterReason> filterReasons)
void onNodeFiltering(NodeFilteringEvent nodeFilteringEvent)

Each INodeFilterReason names the node, its status and the reason. The callback suits structured logging, error reporting pipelines and post-processing analysis. The callback methods are legacy methods: they subscribe to the same events internally and stay supported.

Filter events as an event

The event API is recommended, and the AutoFilter examples use it: getOptimizationEvents().nodeFilteringSubject() is a stream of NodeFilteringEvent objects.

// Each filter reason names the node, its status and the reason.
opti.getOptimizationEvents()
    .nodeFilteringSubject()
    .subscribe(f -> f.getFilterReasons().forEach(r -> System.out.println(r.toString())));

The event form suits event streaming, telemetry integration and status updates in a user interface.

Recommended practice:

• Map INodeFilterReason to your domain identifiers (order ID, customer ID).
• Persist filtered nodes as "unplanned tasks" with reason codes.
• Trigger re-dispatch workflows: manual review, next-day planning, or adding a resource.

Rollout in steps

Step 1. Decide your policy. The AutoFilter is appropriate when the business can accept a feasible plan for most nodes plus a list of excluded tasks. If all tasks must be scheduled, fix the feasibility instead (add resources, relax windows).

Step 2. Start with the selective AutoFilter and one category, typically late arrival. Observe which nodes get filtered and how often.

Step 3. Add margins that reflect the operational tolerance. If 2 minutes late is acceptable, set the late margin to 120 seconds. This prevents unnecessary exclusions.

Step 4. Use node-level constraints for optional tasks. Mark mandatory tasks as protected. Apply filtering constraints only to deferrable work.

Step 5. Use the result in your workflow. Build a standard post-processing contract: planned tasks (routes), filtered tasks (with reasons), and remaining violations (if any soft constraints are allowed).

AutoFilter in the JSON input of the REST service

The REST service takes the same property keys in optimizationOptions.properties of the JSON input, with the values as strings, for example "JOpt.SelectiveAutoFilter": "TRUE". The node field minAutoFilterProtectedExecutions is the JSON form of setMinAutoFilterProtectedExecutions. The API reference shows the schema.

Properties

Optimization properties lists every key below with its default and its allowed values.

Violation categories

The AutoFilter can react to specific types of violations. Each category is switched on or off by its own selective property.

Late arrival, early arrival, working hours and route distance each have a margin that sets a tolerance (see Margins). For example, a late arrival margin of 120 seconds means the AutoFilter only counts a violation if the arrival is more than 2 minutes late. This prevents filtering for negligible deviations that are operationally acceptable.

Certain violation types (such as late or early arrival) can also be configured to filter a node without the statistical ratio, in the strict mode of the last AutoFilter execution. Usually the AutoFilter removes nodes in chunks over several rounds, not all at once.

Switches

Margins

Result events

Limits

• A phase of the optimization scheme with hasAutoFilter(false) runs without the AutoFilter, even if the properties switch it on; see the expert example Algorithm selection.
• With JOpt.SelectiveAutoFilter switched on but no category property switched on and no node-level constraint attached, the AutoFilter filters nothing.
• When every task must be planned, the AutoFilter is the wrong tool: fix the feasibility instead (add resources, relax time windows, adjust capacity, fix travel times and distances).

Related examples

• AutoFilter example page: both AutoFilter examples explained, with configuration templates and a troubleshooting list
• AutoFilterLateExample.java: global selective filtering for late arrivals
• AutoFilterParticularNodeExample.java: a late filter constraint on a single node

Related pages

• Special features: AutoFilter: the AutoFilter section of the special features page
• Optimization properties: every AutoFilter key with its default and allowed values
• Pillar nodes: time windows that the optimizer keeps by architecture

Authors

DNA Evolutions GmbH

Added: 2026-03-29 (JOpt core 7.5.3-j17) | Last updated: 2026-10-23