Sign up

WOQL Set Operations Reference

WOQL provides native set operations with O(n log n) performance, enabling efficient comparison of large datasets. These operations can handle 100,000+ elements in under a second.

See also: Integration tests | CSV comparison guide

Operations Overview

OperationDescriptionComplexity
list_to_setSorts list and removes duplicatesO(n log n)
set_difference(A, B, Result)Elements in A but not in BO(n log n)
set_intersection(A, B, Result)Elements in both A and BO(n log n)
set_union(A, B, Result)All unique elements from A and BO(n log n)
set_member(Element, Set)Check if element is in setO(log n)

list_to_set

Converts a list to a sorted set, removing duplicates.

Example: JavaScript
WOQL.list_to_set("v:my_list", "v:my_set")

set_difference

Returns elements in the first set that are not in the second set.

Example: JavaScript
WOQL.set_difference("v:set_a", "v:set_b", "v:only_in_a")

Example: Finding items to add and remove during synchronization:

Example: JavaScript
WOQL.and(
  WOQL.list_to_set("v:source_list", "v:source_set"),
  WOQL.list_to_set("v:target_list", "v:target_set"),
  WOQL.set_difference("v:source_set", "v:target_set", "v:to_add"),
  WOQL.set_difference("v:target_set", "v:source_set", "v:to_remove"),
)

set_intersection

Returns elements that exist in both sets.

Example: JavaScript
WOQL.set_intersection("v:set_a", "v:set_b", "v:in_both")

set_union

Returns all unique elements from both sets combined.

Example: JavaScript
WOQL.set_union("v:set_a", "v:set_b", "v:combined")

set_member

Checks if an element exists in a set. Returns true/false via backtracking.

Example: JavaScript
WOQL.set_member("v:element", "v:my_set")

Use with not for exclusion checks:

Example: JavaScript
WOQL.not(WOQL.set_member("v:element", "v:excluded_set"))

Performance Benchmarks

Elementsset_differenceset_intersection
1,0007ms5ms
5,00030ms17ms
10,00064ms66ms
100,000577ms445ms

Complete Example: Data Synchronization

Example: JavaScript
WOQL.and(
  // Build source list using group_by
  WOQL.group_by(
    [],
    ["val"],
    "v:source_list",
    WOQL.or(
      WOQL.eq("v:val", WOQL.literal("A", "xsd:string")),
      WOQL.eq("v:val", WOQL.literal("B", "xsd:string")),
      WOQL.eq("v:val", WOQL.literal("C", "xsd:string")),
    )
  ),
  
  // Build target list
  WOQL.group_by(
    [],
    ["val"],
    "v:target_list",
    WOQL.or(
      WOQL.eq("v:val", WOQL.literal("B", "xsd:string")),
      WOQL.eq("v:val", WOQL.literal("C", "xsd:string")),
      WOQL.eq("v:val", WOQL.literal("D", "xsd:string")),
    )
  ),
  
  // Convert to sets
  WOQL.list_to_set("v:source_list", "v:source_set"),
  WOQL.list_to_set("v:target_list", "v:target_set"),
  
  // Compute differences
  WOQL.set_difference("v:source_set", "v:target_set", "v:to_add"),
  WOQL.set_difference("v:target_set", "v:source_set", "v:to_remove"),
  WOQL.set_intersection("v:source_set", "v:target_set", "v:unchanged"),
)

Result:

  • to_add: ["A"] (in source, not in target)
  • to_remove: ["D"] (in target, not in source)
  • unchanged: ["B", "C"] (in both)

Streaming with set_member

For large datasets, use set_member with backtracking to check individual values:

Example: JavaScript
WOQL.and(
  // Load reference set
  WOQL.group_by([], ["val"], "v:ref_list", /* ... */),
  WOQL.list_to_set("v:ref_list", "v:ref_set"),
  
  // Stream through values to check
  WOQL.triple("v:doc", "@schema:id", "v:id"),
  
  // O(log n) membership check
  WOQL.not(WOQL.set_member("v:id", "v:ref_set")),
)

This processes one value at a time, using O(log n) lookups against the set.

Report issue

Was this helpful?