Don't Repeat Yourself, Use Metaprogramming

Writing functions using metaprogramming: AlphaVantage.jl

Returning to the example above, the Alpha Vantage APIs are grouped into four categories: Stock Time Series Data, Physical and Digital/Crypto Currencies, Stock Technical Indicators, and Sector Performances. However, there is actually only a single endpoint that accepts a query parameter, function, that determines actual data request. This function also determines which other query parameters are required.

Because different data requests may require different query parameters there is an AlphaVantage.jl function for every Alpha Vantage function. However many functions share the same arguments. For example, the time_series_daily function accepts the same arguments as time_series_daily_adjusted and time_series_weekly.

It would be cumbersome and inefficient to define each function manually. Fortunately this is easy with metaprogramming – generating the functions through code.

for func in (:intraday, :daily, :weekly, :monthly)
    x = "digital_currency_$(func)"
    fname = Symbol(x)
    @eval begin
        function ($fname)(symbol::String; market::String="USD", outputsize::String="compact", datatype::String="json")
            @argcheck in(outputsize, ["compact", "full"])
            @argcheck in(datatype, ["json", "csv"])
            uri = "$(alphavantage_api)query?function=" *
                uppercase($x) *
                "&symbol=$symbol&market=$market&outputsize=$outputsize&datatype=$datatype&apikey=" *
                ENV["ALPHA_VANTAGE_API_KEY"]
            data = _get_request(uri)
            return _parse_response(data, datatype)
        end
        export $fname
    end
end

This code loops through an array of symbols (representing the code as data). For each symbol, the code creates a function that accepts several arguments and makes an API request. The trick here is that the function query parameter is assigned a value using the symbol. Within the @eval block the symbols are evaluated to their corresponding representation (e.g. when $func is intraday then $fname becomes digital_currency_intraday).

Benchmarking a package using metaprogramming: AdventOfCode2017.jl

Another example of metaprogramming in Julia is in the AdventOfCode2017.jl README. Advent of Code is a collection of small programming problems. New problems are released each year during the first 25 days of December.

AdventOfCode2017.jl contains my Julia solutions to the puzzles. The README is generated from README.jl using Weave–a package for converting Julia to markdown (similar to knitr or Sweave in R)–and contains performance benchmarks for each puzzle solution.

The first few lines of a non-metaprogramming approach might look something like this:

@time puzzle01()
@time puzzle02()
@time puzzle03()
@time puzzle04()
...

In this case the code violates the DRY principle. Fortunately metaprogramming provides tools to express this more succinctly. The code from README.jl is below:

table = ["Day" "Elapsed (s)" "Allocated (MiB)" "Garbage Collection (s)"]
puzzles = filter(x-> x≠ :AdventOfCode2017, names(AdventOfCode2017, false))
for puzzle in puzzles
    m = match(r"[0-9]+", String(puzzle))
    day = string(parse(Int, m.match))
    @eval begin
        val, t, bytes, gctime, memallocs = @timed $puzzle()
        t = round(t, 4)
        kib = round(bytes/1024^2, 4)
        gctime = round(gctime, 4)
        row = [$day t kib gctime]
        table = vcat(table, row)
    end
end

Instead of just measuring performance of each function using @time, I decided to store results in a table. The names function returns an array of symbols for each function exported by the package. Then for each symbol, I measure the performance of the function and append that row to the table.

Metaprogramming can help you stay DRY

Metaprogramming is code (a program) that writes more code (another program). Under the hood, Julia represents its code as a data structure and this can be edited programmatically. For example, you can use metaprogramming to write functions that take a set of similar arguments or evaluate many functions to measure their performance.