Interface

Backwards(something)

Reverse sorting order.

julia> using LightQuery


julia> using Test: @inferred


julia> collect(@inferred order([1, -2], Backwards))
2-element Array{Int64,1}:
  1
 -2

By(sorted, key_function)

Mark that sorted has been pre-sorted by key_function. Use with Group, and various joins.

julia> using LightQuery


julia> using Test: @inferred


julia> @inferred By([1, -2], abs)
By{Array{Int64,1},typeof(abs)}([1, -2], abs)

Enumerate{Unenumerated}

"Sees through" most iterators into their parent.

julia> using LightQuery


julia> using Test: @inferred


julia> @inferred collect(Enumerate(Iterators.filter(iseven, [4, 3, 2, 1])))
2-element Array{Tuple{Int64,Int64},1}:
 (1, 4)
 (3, 2)

Group(ungrouped::By)

Group consecutive items in ungrouped.

julia> using LightQuery


julia> using Test: @inferred


julia> @inferred collect(Group(By([1, -1, -2, 2, 3, -3], abs)))
3-element Array{Tuple{Int64,SubArray{Int64,1,Array{Int64,1},Tuple{UnitRange{Int64}},true}},1}:
 (1, [1, -1])
 (2, [-2, 2])
 (3, [3, -3])

julia> (@inferred collect(Group(By(Int[], abs)))) == []
true

Requires a presorted object (see By); order first if not.

julia> using LightQuery


julia> using Test: @inferred


julia> first_group = @> [2, 1, 2, 1] |> order(_, identity) |> Group(By(_, identity)) |> first;


julia> key(first_group)
1

julia> collect(value(first_group))
2-element Array{Int64,1}:
 1
 1

InnerJoin(left::By, right::By)

Find all pairs where isequal(left.key_function(left.sorted), right.key_function(right.sorted)). Assumes left and right are both strictly sorted (no repeats). If there are repeats, Group first. See By.

julia> using LightQuery


julia> using Test: @inferred


julia> @inferred collect(InnerJoin(By([1, -2, 5, -6], abs), By([-1, 3, -4, 6], abs)))
2-element Array{Tuple{Int64,Int64},1}:
 (1, -1)
 (-6, 6)

julia> (@inferred collect(InnerJoin(By(Int[], abs), By(Int[], abs)))) == []
true

julia> (@inferred collect(InnerJoin(By([1], abs), By(Int[], abs)))) == []
true

julia> (@inferred collect(InnerJoin(By(Int[], abs), By([1], abs)))) == []
true

LeftJoin(left::By, right::By)

Find all pairs where isequal(left.key_function(left.sorted), right.key_function(right.sorted)), using missing when there is no right match. Assumes left and right are both strictly sorted (no repeats). If there are repeats, Group first. See By.

julia> using LightQuery


julia> collect(LeftJoin(By([1, -2, 5, -6], abs), By([-1, 3, -4, 6], abs)))
4-element Array{Tuple{Int64,Union{Missing, Int64}},1}:
 (1, -1)
 (-2, missing)
 (5, missing)
 (-6, 6)

julia> collect(LeftJoin(By(Int[], abs), By(Int[], abs))) == []
true

julia> collect(LeftJoin(By([1], abs), By(Int[], abs)))
1-element Array{Tuple{Int64,Union{Missing, Int64}},1}:
 (1, missing)

julia> collect(LeftJoin(By(Int[], abs), By([1], abs))) == []
true

Length(iterator, new_length)

Allow optimizations based on length.

julia> using LightQuery


julia> collect(Length(Iterators.filter(iseven, 1:4), 2))
2-element Array{Int64,1}:
 2
 4

Name(name)

Create a typed Name. Inverse of unname. See also @name_str.

julia> using LightQuery


julia> Name(:a)
name"a"

Names can be used as selector functions.

julia> using Test: @inferred


julia> data = (a = 1, b = 1.0, c = 1, d = 1.0, e = 1, f = 1.0)
(a = 1, b = 1.0, c = 1, d = 1.0, e = 1, f = 1.0)

julia> @inferred name"c"(data)
1

Multiple names can be used as selector functions

julia> @inferred (name"c", name"f")(data)
(c = 1, f = 1.0)

A final use for names can be as a way to construct NamedTuples from pairs. Using Names instead of Symbols gives type stability.

julia> @inferred NamedTuple((name"a", 1), (name"b", 2))
(a = 1, b = 2)

OuterJoin(left::By, right::By)

Find all pairs where isequal(left.key_function(left.sorted), right.key_function(right.sorted)), using missing when there is no left or right match. Assumes left and right are both strictly sorted (no repeats). If there are repeats, Group first. See By.

julia> using LightQuery


julia> collect(OuterJoin(By([1, -2, 5, -6], abs), By([-1, 3, -4, 6], abs)))
6-element Array{Tuple{Union{Missing, Int64},Union{Missing, Int64}},1}:
 (1, -1)
 (-2, missing)
 (missing, 3)
 (missing, -4)
 (5, missing)
 (-6, 6)

julia> collect(OuterJoin(By(Int[], abs), By(Int[], abs))) == []
true

julia> collect(OuterJoin(By([1], abs), By(Int[], abs)))
1-element Array{Tuple{Union{Missing, Int64},Union{Missing, Int64}},1}:
 (1, missing)

julia> collect(OuterJoin(By(Int[], abs), By([1], abs)))
1-element Array{Tuple{Union{Missing, Int64},Union{Missing, Int64}},1}:
 (missing, 1)

Peek(rows, maximum_length = 4)

Peek an iterator which returns named tuples. Will show no more than maximum_length rows.

julia> using LightQuery


julia> Peek(Rows(a = 1:5, b = 5:-1:1))
Showing 4 of 5 rows
|   a |   b |
| ---:| ---:|
|   1 |   5 |
|   2 |   4 |
|   3 |   3 |
|   4 |   2 |

RightJoin(left::By, right::By)

Find all pairs where isequal(left.key_function(left.sorted), right.key_function(right.sorted)), using missing when there is no left match. Assumes left and right are both strictly sorted (no repeats). If there are repeats, Group first. See By.

julia> using LightQuery


julia> collect(RightJoin(By([1, -2, 5, -6], abs), By([-1, 3, -4, 6], abs)))
4-element Array{Tuple{Union{Missing, Int64},Int64},1}:
 (1, -1)
 (missing, 3)
 (missing, -4)
 (-6, 6)

julia> collect(RightJoin(By(Int[], abs), By(Int[], abs))) == []
true

julia> collect(RightJoin(By([1], abs), By(Int[], abs))) == []
true

julia> collect(RightJoin(By(Int[], abs), By([1], abs)))
1-element Array{Tuple{Union{Missing, Int64},Int64},1}:
 (missing, 1)

Rows(named_columns)

Iterator over rows of a table. Always lazy. Use Peek to view.

julia> using LightQuery


julia> using Test: @inferred


julia> lazy = @inferred Rows(a = [1, 2], b = [1.0, 2.0])
2-element Rows{NamedTuple{(:a, :b),Tuple{Int64,Float64}},1,Tuple{Array{Int64,1},Array{Float64,1}},Tuple{Name{:a},Name{:b}}}:
 (a = 1, b = 1.0)
 (a = 2, b = 2.0)

julia> @inferred collect(lazy)
2-element Array{NamedTuple{(:a, :b),Tuple{Int64,Float64}},1}:
 (a = 1, b = 1.0)
 (a = 2, b = 2.0)

julia> @inferred Rows(a = [1, 2])
2-element Rows{NamedTuple{(:a,),Tuple{Int64}},1,Tuple{Array{Int64,1}},Tuple{Name{:a}}}:
 (a = 1,)
 (a = 2,)

All arguments to Rows must have the same axes. Use @inbounds to override the check.

julia> result = Rows(a = 1:2, b = 1:3)
ERROR: DimensionMismatch("All columns passed to `Rows` must have the same axes")
[...]

gather(data, new_name_old_names...)

For each new_name, old_names pair in new_name_old_names, gather the old_names into a single new_name. Inverse of spread.

julia> using LightQuery


julia> using Test: @inferred


julia> data = (a = 1, b = 1.0, c = 1, d = 1.0, e = 1, f = 1.0);


julia> @inferred gather(data, g = (name"b", name"e"), h = (name"c", name"f"))
(a = 1, d = 1.0, g = (b = 1.0, e = 1), h = (c = 1, f = 1.0))

index(unindexed, key_function)

Index unindexed by the results of key_function. Results of key_function must be unique.

julia> using LightQuery


julia> using Test: @inferred


julia> result = @inferred index([-2, 1], abs)
LightQuery.Indexed{Int64,Int64,Array{Int64,1},Dict{Int64,Int64}} with 2 entries:
  2 => -2
  1 => 1

julia> @inferred result[2]
-2

key(pair)

The key in a key => value pair.

julia> using LightQuery


julia> using Test: @inferred


julia> @inferred key(:a => 1)
:a

julia> @inferred key((:a, 1))
:a

make_columns(rows)

Collect into columns.

julia> using LightQuery

julia> import Compat: Iterators

julia> using Test: @inferred


julia> stable(x) = (a = x, b = x + 0.0, c = x, d = x + 0.0, e = x, f = x + 0.0);


julia> @inferred make_columns(Iterators.map(stable, 1:4))
(a = [1, 2, 3, 4], b = [1.0, 2.0, 3.0, 4.0], c = [1, 2, 3, 4], d = [1.0, 2.0, 3.0, 4.0], e = [1, 2, 3, 4], f = [1.0, 2.0, 3.0, 4.0])

julia> unstable(x) =
           if x <= 2
               (a = missing, b = string(x), d = string(x))
           else
               (a = x, b = missing, c = x)
           end;


julia> make_columns(Iterators.map(unstable, 1:4))
(d = Union{Missing, String}["1", "2", missing, missing], a = Union{Missing, Int64}[missing, missing, 3, 4], b = Union{Missing, String}["1", "2", missing, missing], c = Union{Missing, Int64}[missing, missing, 3, 4])

julia> make_columns(Iterators.filter(row -> true, Iterators.map(unstable, 1:4)))
(d = Union{Missing, String}["1", "2", missing, missing], a = Union{Missing, Int64}[missing, missing, 3, 4], b = Union{Missing, String}["1", "2", missing, missing], c = Union{Missing, Int64}[missing, missing, 3, 4])

order(unordered, key_function; keywords...)

Generalized sort. keywords will be passed to sort!; see the documentation there for options. Use By to mark that an object has been sorted. If the results of key_function are type unstable, consider using hash ∘ key_function instead.

julia> using LightQuery


julia> using Test: @inferred


julia> collect(@inferred order([-2, 1], abs))
2-element Array{Int64,1}:
  1
 -2

function reduce_rows(rows, a_function, columns...)

Reduce a function over each of columns in rows.

julia> using LightQuery


julia> using Test: @inferred


julia> @inferred reduce_rows(Rows(a = [1, 1], b = [1.0, 1.0]), +, name"a", name"b")
(a = 2, b = 2.0)

remove(data, old_names...)

Remove old_names from data.

julia> using LightQuery


julia> using Test: @inferred


julia> data = (a = 1, b = 1.0, c = 1, d = 1.0, e = 1, f = 1.0);


julia> @inferred remove(data, name"c", name"f")
(a = 1, b = 1.0, d = 1.0, e = 1)

rename(data, new_name_old_names...)

Rename data.

julia> using LightQuery


julia> using Test: @inferred


julia> data = (a = 1, b = 1.0, c = 1, d = 1.0, e = 1, f = 1.0);


julia> @inferred rename(data, c2 = name"c", f2 = name"f")
(a = 1, b = 1.0, d = 1.0, e = 1, c2 = 1, f2 = 1.0)

spread(data, some_names...)

Unnest nested named tuples. Inverse of gather.

julia> using LightQuery


julia> using Test: @inferred


julia> gathered = (a = 1, d = 1.0, g = (b = 1.0, e = 1), h = (c = 1, f = 1.0));


julia> @inferred spread(gathered, name"g", name"h")
(a = 1, d = 1.0, b = 1.0, e = 1, c = 1, f = 1.0)

transform(data, assignments...)

Merge assignments into data, overwriting old values.

julia> using LightQuery


julia> using Test: @inferred


julia> data = (a = 1, b = 1.0, c = 1, d = 1.0, e = 1, f = 1.0);


julia> @inferred transform(data, c = 2.0, f = 2, g = 1, h = 1.0)
(a = 1, b = 1.0, d = 1.0, e = 1, c = 2.0, f = 2, g = 1, h = 1.0)

unname(::Name{name}) where name

Inverse of Name.

julia> using LightQuery


julia> using Test: @inferred


julia> @inferred unname(Name(:a))
:a

value(pair)

The value in a key => value pair.

julia> using LightQuery


julia> using Test: @inferred


julia> @inferred value(:a => 1)
1

julia> @inferred value((:a, 1))
1

macro >(body)

If body is in the form object_ |> call_, call @_ on call, and recur on object.

julia> using LightQuery


julia> @> 0 |> _ - 1 |> abs |> Base.abs
1

You can't include multiple arguments.

julia> @> _ |> _ + __
ERROR: LoadError: ArgumentError: Chain segments must contain single underscores only
[...]

You can nest chains:

julia> @> 1 |> (@> _ + 1 |> _ + 1)
3

Handles interpolations seamlessly:

julia> @> 1 |> :(_ + $_)
:(_ + 1)

macro _(body)

Terser function syntax. The arguments are inside the body; the first argument is _, the second argument is __, etc. Will @inline.

julia> using LightQuery


julia> using Test: @inferred


julia> @inferred (@_ _ + 1)(1)
2

julia> @inferred map((@_ __ - _), (1, 2), (2, 1))
(1, -1)

If there are no _ arguments, read as is.

julia> (@_ x -> x + 1)(1)
2

@if_known(something)

If something is missing, return missing, otherwise, something.

julia> using LightQuery


julia> using Test: @inferred


julia> function test(x)
           first(@if_known(x))
       end;


julia> @inferred test((1, 2))
1

julia> @inferred test(missing)
missing

name_str(name)

Create a typed Name.

julia> using LightQuery


julia> name"a"
name"a"