Go version 1.23 added iterator support 1 and the iter package
2. We can now loop
over constants, containers (maps, slices,
arrays,
strings) and functions. At first I found the iterator creation clunky,
while consuming the iterator seems straightforward.
My issue with the go way of iterators is, you can’t chain them like you would in JavaScript:
1[1,2,3,4]
2 .reverse()
3 .map(e => e*e)
4 .filter(e => e % 2 == 0)
5 .forEach(e => console.log(e))
The Annoyance
Writing this in go would require us to chain 5 function calls:
1slices.ForEach(
2 slices.Filter(
3 slices.Map(
4 slices.Reverse(slices.All([]int{1,2,3,4})),
5 func(i int) int { return i * i},
6 ),
7 func(i int) bool { return i % 2 == 0 }
8 ),
9 func(i int) { fmt.Println(i) }
10)
This is an example, there are are no Map, Filter or ForEach functions in the slices package 3.
The Solution (sort of)
Because i have a big distaste for writing chained “functional” operations like this, looking at you python (don’t come at me haskell bros) - I wanted to use the new iterators and the iter package and wrap it with a structure to allow the nice and clean chaining JavaScript provides. Below are the same operations, but instead of using the iter and slices package, I use my abstraction:
1func TestIterator(t *testing.T) {
2 From([]int{1, 2, 3, 4}).
3 Reverse().
4 Map(func(i int) int { return i * i }).
5 Filter(func(i int) bool { return i%2 == 0 }).
6 Each(func(a int) { println(a) })
7 // 16
8 // 4
9}
The Logic
Lets take a look a the implementation and let me introduce the Iteratorstruct.
It wraps the iterator (*Iterator).iter and thus allows me to callfunctions on
this structure, instead of having to use each iterator functionas a param to the
next one.
1type Iterator[V any] struct {
2 iter iter.Seq[V]
3}
Lets take a look at the first functions coming to mind when talking about iterators, creating one from a slice and collection one into a slice:
1func (i Iterator[V]) Collect() []V {
2 collect := make([]V, 0)
3 for e := range i.iter {
4 collect = append(collect, e)
5 }
6 return collect
7}
8
9func From[V any](slice []V) *Iterator[V] {
10 return &Iterator[V]{
11 iter: func(yield func(V) bool) {
12 for _, v := range slice {
13 if !yield(v) {
14 return
15 }
16 }
17 },
18 }
19}
The first one is as straight forward as possible - create a slice, consume the
iterator, append each element, return the slice. The second highlights the weird
way iterators are created in go. Lets first take a look at the signature, we are
returning a pointer to the struct, so the callee can invoke all methods without
having to use a temporary variable for each. In the function itself, the
iterator is created by returning a closure, that loops over the param and
returning, which stops the iterator, when the yield function returns false.
Each
The ForEach / Each method is the next function I want, It simply executes
the passed in function for every element of the iterator.
1func (i *Iterator[V]) Each(f func(V)) {
2 for i := range i.iter {
3 f(i)
4 }
5}
It can be used like this:
1From([]int{1, 2, 3, 4}).Each(func(a int) { println(a) })
2// 1
3// 2
4// 3
5// 4
Reverse
A way to reverse the iterator, we first need to collect all elements and after that construct a new iterator from the collected slice, luckily we have functions for exactly this:
1func (i *Iterator[V]) Reverse() *Iterator[V] {
2 collect := i.Collect()
3 counter := len(collect) - 1
4 for e := range i.iter {
5 collect[counter] = e
6 counter--
7 }
8 return From(collect)
9}
Again, useful to reverse a slice:
1From([]int{1, 2, 3, 4}).Reverse().Each(func(a int) { println(a) })
2// 4
3// 3
4// 2
5// 1
Map
Mutating every element of the iterator is a necessity, too:
1func (i *Iterator[V]) Map(f func(V) V) *Iterator[V] {
2 cpy := i.iter
3 i.iter = func(yield func(V) bool) {
4 for v := range cpy {
5 v = f(v)
6 if !yield(v) {
7 return
8 }
9 }
10 }
11 return i
12}
At first we copy the previous iterator, by doing this, we skip causing a stack
overflow by referencing the i.iter iterator in the iterator itself. Map is
works by overwriting the i.iter with a new iterator thats consuming every
field of the cpy iterator and overwriting the iterator value with the result
of passing v to f, thus mapping over the iterator.
Filter
After mapping, possibly the most used streaming / functional api method is
Filter. So lets take a look at our final operation:
1func (i *Iterator[V]) Filter(f func(V) bool) *Iterator[V] {
2 cpy := i.iter
3 i.iter = func(yield func(V) bool) {
4 for v := range cpy {
5 if f(v) {
6 if !yield(v) {
7 return
8 }
9 }
10 }
11 }
12 return i
13}
Similar to Map, we consume the copied iterator and invoke f with v as the
param for each one, if f returns true, we keep it in the new iterator.
Examples and Thoughts
The slices and the iter package both play very good together with the generic system introduced in go 1.18 4.
While this API is easier to use, I understand the reasoning of the go team for not implementing iterators like this. Below are some tests that serve as examples and the result of running them.
1package iter1
2
3import (
4 "fmt"
5 "testing"
6 "unicode"
7)
8
9func TestIteratorNumbers(t *testing.T) {
10 From([]int{1, 2, 3, 4}).
11 Reverse().
12 Map(func(i int) int { return i * i }).
13 Filter(func(i int) bool { return i%2 == 0 }).
14 Each(func(a int) { println(a) })
15}
16
17func TestIteratorRunes(t *testing.T) {
18 r := From([]rune("Hello World!")).
19 Reverse().
20 // remove all spaces
21 Filter(func(r rune) bool { return !unicode.IsSpace(r) }).
22 // convert every rune to uppercase
23 Map(func(r rune) rune { return unicode.ToUpper(r) }).
24 Collect()
25 fmt.Println(string(r))
26}
27
28func TestIteratorStructs(t *testing.T) {
29 type User struct {
30 Id int
31 Name string
32 Hash int
33 }
34
35 u := []User{
36 {0, "xnacly", 0},
37 {1, "hans", 0},
38 {2, "gedigedagedeio", 0},
39 }
40
41 From(u).
42 // computing the hash for each user
43 Map(func(u User) User {
44 h := 0
45 for i, r := range u.Name {
46 h += int(r)*31 ^ (len(u.Name) - i - 1)
47 }
48 u.Hash = h
49 return u
50 }).
51 Each(func(u User) { fmt.Printf("%#+v\n", u) })
52}
Running these, results in:
1$ go test ./... -v
2=== RUN TestIteratorNumbers
316
44
5--- PASS: TestIteratorNumbers (0.00s)
6=== RUN TestIteratorRunes
7!DLROWOLLEH
8--- PASS: TestIteratorRunes (0.00s)
9=== RUN TestIteratorStructs
10&iter1.User{Id:0, Name:"xnacly", Hash:20314}
11&iter1.User{Id:1, Name:"hans", Hash:13208}
12&iter1.User{Id:2, Name:"gedigedagedeio", Hash:44336}
13--- PASS: TestIteratorStructs (0.00s)
14PASS
15ok iter1 0.263s
So there u have it, a wrapper around iter and slices to mirror the way
JavaScript provides streaming, only in go.