Having fun with Go in maru2
TL;DR maru2 is a simple and modern task runner. As such it gets the benefit of being written with the most current language features in Go. Some of these are small UX/DX improvements, while others greatly reduce boilerplate and increase a reader's comprehension of the code.
Iterators
Added in Go 1.23, iterators allow for traditionally non-range
able types to be iterated over.
In maru2, iterators are used to create ordered sequences for map types when traditional range
would produce an unordered set.
// schema/v1/task.go
type TaskMap map[string]Task
func (tm TaskMap) OrderedTaskNames() []string {
names := make([]string, 0, len(tm))
for k := range tm {
names = append(names, k)
}
slices.SortStableFunc(names, func(a, b string) int {
if a == schema.DefaultTaskName {
return -1
}
if b == schema.DefaultTaskName {
return 1
}
return cmp.Compare(a, b)
})
return names
}
func (tm TaskMap) OrderedSeq() iter.Seq2[string, Task] {
names := tm.OrderedTaskNames()
return func(yield func(string, Task) bool) {
for _, name := range names {
task := tm[name]
if !yield(name, task) {
return
}
}
}
}
This allows for drop-in replacement of for ... range
syntax depending upon if the operation needs to be ordered or not:
for name, task := range wf.Tasks.OrderedSeq() {}
// vs
for name, task := range wf.Tasks {}
Compile time interface checking
In maru2, we have the --fetch-policy
flag that controls the caching behavior of workflows. Since fetch policy is a string enum, we need to implement the pflag.Value
interface to ensure we can use it natively as a cobra.VarP
var _ pflag.Value = (*FetchPolicy)(nil)
Interface implementations in go are implicit. If a type has a String() string
method, it implements fmt.Stringers
.
Sometimes, especially when first developing a type that must implement an interface, and especially when that interface is from another module, it can be beneficial to ensure a concrete construction of that type fully satisfies the interface.
The above syntax creates an "empty" variable using a blank identifier whose type is declared to satisfy pflag.Value
and whose current value is a nil construction of the FetchPolicy
type.
This allows for a slightly cleaner error when the implementation and interface drift as package var
s are evaluated at compile time.
Fallthrough
When caching, maru2 follows different behavior based upon the FetchPolicy
enum. However, certain behavior is not exclusive to a single policy.
// uses/store_fetcher.go
// StoreFetcher is a fetcher that wraps another fetcher and caches the results
// in a store according to the cache policy.
type StoreFetcher struct {
Source Fetcher
Store Storage
Policy FetchPolicy
}
// Fetch implements the Fetcher interface
func (f *StoreFetcher) Fetch(ctx context.Context, uri *url.URL) (io.ReadCloser, error) {
switch f.Policy {
case FetchPolicyNever:
return f.Store.Fetch(ctx, uri)
case FetchPolicyIfNotPresent:
exists, err := f.Store.Exists(uri)
if err != nil {
return nil, err
}
if exists {
return f.Store.Fetch(ctx, uri)
}
fallthrough // <-- LOOK HERE
case FetchPolicyAlways:
rc, err := f.Source.Fetch(ctx, uri)
if err != nil {
return nil, err
}
defer rc.Close()
if err := f.Store.Store(rc, uri); err != nil {
return nil, err
}
return f.Store.Fetch(ctx, uri)
default:
return nil, fmt.Errorf("unsupported fetch policy: %s", f.Policy)
}
}
Sometimes multiple switch cases can be correct, but you desire the behavior of one case before the other is triggered, this is where fallthrough
shines.
https://go.dev/ref/spec#Fallthrough_statements
In the above, when fetching from the store, we save duplicated logic when fetching content that has not been cached yet.
min and max
Go has builtin min
and max
functions for arithmetic operations.
https://go.dev/ref/spec#Min_and_max
When duplicating the behavior of just --list
in maru2 --list
. I noticed that it was not a true table.
$ just --list
...
clean # Clean Repo
generate-build-tags image="bluefin" tag="latest" flavor="main" kernel_pin="" ghcr="0" $version="" github_event="" github_number="" # Generate Tags
generate-default-tag tag="latest" ghcr="0" # Generate Default Tag
secureboot $image="bluefin" $tag="latest" $flavor="main" # Secureboot Check
tag-images image_name="" default_tag="" tags="" # Tag Images
Notice in the above, it is a best effort alignment of the command and args to the description comment.
In maru2, we use min
and max
to replicate this behavior so that we try our best to keep everything aligned, whilst preventing very long content from creating unwieldy gaps.
// log.go
type TaskList struct {
col0max int
rows [][2]string
}
func (tl *TaskList) Row(col0, col1 string) {
// track the current max length of the command string
tl.col0max = max(tl.col0max, ansi.StringWidth(col0))
tl.rows = append(tl.rows, [2]string{col0, col1})
}
func (tl *TaskList) String() string {
sb := strings.Builder{}
cutoff := 50 // best effort max length, borrowed from just
for _, row := range tl.rows {
col0, col1 := row[0], row[1]
col0len := ansi.StringWidth(col0)
text0 := lipgloss.NewStyle().MarginLeft(4).Render(col0)
text1 := lipgloss.NewStyle().Foreground(InfoColor).Render(col1)
sb.WriteString(text0)
if col0len > cutoff {
sb.WriteString(text1 + "\n")
} else {
// calculate the padding between this command string and the max length, none if it is above the cutoff
numspaces := min(50-col0len, tl.col0max-col0len)
if numspaces == 0 {
numspaces = 1
}
sb.WriteString(strings.Repeat(" ", numspaces) + text1 + "\n")
}
}
return sb.String()
}
$ maru2 --list
...
echo -w text='Hello, world!' # A simple hello world
hello-world # Used by the benchmark task
test -w e2e='false' -w short='false' -w update-scripts='false' # Run maru2 test suite with coverage
Labeled loops
https://go.dev/ref/spec#Labeled_statements
When garbage collecting "dead" workflows in maru2's store (cache), a labeled loop provides the ability to more quickly iterate over the file system.
// uses/store.go
func (s *LocalStore) GC() error {
s.mu.Lock()
defer s.mu.Unlock()
// list all of the files in the current store
all, err := afero.ReadDir(s.fsys, ".")
if err != nil {
return err
}
// label this loop as "outer"
outer:
for _, fi := range all { // range over all of the files
// skip directories and the index
if fi.IsDir() || fi.Name() == IndexFileName {
continue
}
// for each file, iterate over the in-memory index
for _, desc := range s.index {
if desc.Hex == fi.Name() {
continue outer // if the file matches a value in the current index, abort and continue the "outer" loop, this escapes the current loop iterating over the index entries
}
}
if err := s.fsys.Remove(fi.Name()); err != nil {
return err
}
}
return nil
}
How have you been using underutilized and underappreciated features of Go?