Where The FUCK Are slice.Flatten And slice.Map

Table of Contents

I recently found myself having to work with some 2d slices in Go, when it hit me, go’s slices package has no Flatten or Map functions. I mainly needed to extract these slices from map’s, sort them, filter and pass them to other iterator based slice functions (iter and slices).

GO
 1var sets = map[string][]string{
 2	"nato": {
 3		"Alfa",
 4        "Bravo",
 5        // ...
 6		"Zulu",
 7	},
 8	"german": {
 9		"Anton",
10		"Berta",
11        // ...
12		"Zacharias",
13	},
14}
15
16func main() {
17	// phonetic alphabet stuff
18
19	// there are 26 chars in an alphabet and each set is an alphabet
20	flattend := make([]int, len(sets)*26)
21	idx := 0
22	for inner := range maps.Values(sets) {
23		for _, innerVal := range inner {
24			l := len(innerVal)
25			if l != 0 {
26				flattend[idx] = l
27				idx += 1
28			}
29		}
30	}
31	sum := 0
32	slices.Sort(flattend)
33	for _, v := range flattend {
34		sum += v
35	}
36	fmt.Println(sum)
37}
Terminal
$ go run .
300

Where is Flatten (+Map) and Why even Flatten

Most, if not all, slice functions accept iter.Seq:

GO
1// Seq is an iterator over sequences of individual values.
2// When called as seq(yield), seq calls yield(v) for each value v in the sequence,
3// stopping early if yield returns false.
4// See the [iter] package documentation for more details.
5type Seq[V any] func(yield func(V) bool)

The issue is: converting any 2d slice ([][]T) to an iterator is generally done via slices.All, which yields iter.Seq2

GO
 1// All returns an iterator over index-value pairs in the slice
 2// in the usual order.
 3func All[Slice ~[]E, E any](s Slice) iter.Seq2[int, E] {
 4	return func(yield func(int, E) bool) {
 5		for i, v := range s {
 6			if !yield(i, v) {
 7				return
 8			}
 9		}
10	}
11}
GO
1// Seq2 is an iterator over sequences of pairs of values, most commonly key-value pairs.
2// When called as seq(yield), seq calls yield(k, v) for each pair (k, v) in the sequence,
3// stopping early if yield returns false.
4// See the [iter] package documentation for more details.
5type Seq2[K, V any] func(yield func(K, V) bool)

For instance, passing [][]string to slices.All computes the following generic:

GO
1// func[Slice ~[]E, E any](s Slice) iter.Seq2[int, E]
2func slices.All(s [][]string) iter.Seq2[int, []string]

And none of the below functions accept the above iter.Seq2:

GO
1func Collect[E any](seq iter.Seq[E]) []E
2func AppendSeq[Slice ~[]E, E any](s Slice, seq iter.Seq[E]) Slice
3func Sorted[E cmp.Ordered](seq iter.Seq[E]) []E
4func SortedFunc[E any](seq iter.Seq[E], cmp func(E, E) int) []E
5func SortedStableFunc[E any](seq iter.Seq[E], cmp func(E, E) int) []E

So I would need to consume each iterator value and consume all its elements before moving the resulting temporary container into yet another iterator to pass into all other, iterator consuming functions.

Rust has Iterator.flatten and every other “functional like” language constructs in not so functional languages (JavaScript, etc.) contain them too.

RUST
1// [5, 5]
2vec![vec!["hello"], vec!["world"]]
3    .iter()
4    .flatten()
5    .map(|word| word.len())
6    .collect::<Vec<_>>();
JS
1[["hello"], ["world"]].flat().map(word => word.length);

So I’m asking: Where the fuck are slices.Flatten and slices.Map?:

GO
 1// []string{"hello", "world"}
 2slices.Collect(
 3    slices.Map(
 4        slices.Flatten(
 5            slices.All([][]string{{"hello"}, {"world"}}),
 6        ),
 7        func(in string) int {
 8            return len(in)
 9        },
10    ),
11)

On an whole other note: map.Values from the example returns iter.Seq while slices.All returns iter.Seq2, so we will need to write two Flatten implementations due to Go having a not really generic generic system.

Also this sucks to look at, but I already wrote an article about that: Fun with Go Iterators.

A Flatten, a simpler Flatten and an even simpler Map

Answering the above question:

Even though Go iterators are really weird to write, but easy to consume, these are trivial to understand:

GO
 1// Map takes a Seq[T] and applies fn to each element, producing a Seq[V].
 2func Map[T any, V any](seq iter.Seq[T], fn func(T) V) iter.Seq[V] {
 3	return func(yield func(V) bool) {
 4		seq(func(t T) bool {
 5			v := fn(t)
 6			return yield(v)
 7		})
 8	}
 9}
10
11// Flatten flattens Seq[[]T] into Seq[T]
12func Flatten[T any](outer iter.Seq[[]T]) iter.Seq[T] {
13	return func(yield func(T) bool) {
14		outer(func(inner []T) bool {
15			for _, val := range inner {
16				if !yield(val) {
17					return false
18				}
19			}
20			return true
21		})
22	}
23}
24
25// FlattenSeq2 flattens Seq2[K, []T] into Seq[T]
26func FlattenSeq2[T any, K any](outer iter.Seq2[K, []T]) iter.Seq[T] {
27	return func(yield func(T) bool) {
28		outer(func(_ K, inner []T) bool {
29			for _, val := range inner {
30				if !yield(val) {
31					return false
32				}
33			}
34			return true
35		})
36	}
37}

Reworking the example we started out with with the new Flatten and Map iterator functions:

GO
 1func main() {
 2	sum := 0
 3	for _, v := range slices.Sorted(
 4		Map(
 5			Flatten(maps.Values(sets)),
 6			func(in string) int {
 7				return len(in)
 8			},
 9		),
10	) {
11		sum += v
12	}
13	fmt.Println(sum)
14}

And using Flatten with slices.All via

GO
1FlattenSeq2(slices.All([][]string{}))

For example:

GO
 1// gen produces `amount` of tuples of len `tupleSize` with unique
 2// members of sets
 3func gen(sets [][]string, amount int, tupleSize int) [][]string {
 4	allWords := slices.Collect(FlattenSeq2(slices.All(sets)))
 5	allWordsLen := len(allWords)
 6	r := make([][]string, amount)
 7	for i := range amount {
 8		allWordsLocal := make([]string, allWordsLen)
 9		copy(allWordsLocal, allWords)
10		r[i] = make([]string, toupleSize)
11		for j := range toupleSize {
12			idx := rand.Intn(len(allWordsLocal) - 1)
13			r[i][j] = allWordsLocal[idx]
14			allWordsLocal[idx] = allWordsLocal[len(allWordsLocal)-1]
15			allWordsLocal = allWordsLocal[:len(allWordsLocal)-1]
16		}
17	}
18	return r
19}

Where are the other iterator helpers and why are Go iterators like this?

Go iterators are probably a good idea? I don’t think they fit into the language at all and they lack so many features that I perceive them to be somewhat unfinished. Especially compared to something like the interface system and how most packages feel designed around the idea of plug and play Writer and Reader interaction. Maybe I am too spoiled by rust and pre generic Go.

Since I’m already here, where are:

  1. Filter
  2. FlatMap
  3. FilterMap
  4. Sum
  5. …

Also, I don’t have (or want) a google account so I won’t contribute any of these, nor Flatten or Map, to the Go project - even disregarding the fact that they probably wouldn’t be accepted as part of slices.