Bringing order to chaos with Jason

I recently had an issue where Videos stopped loading on the project we were working on, but all other developers on the team, Review and Production didn’t have this issue.

It was difficult to track down, but ultimately it was down to the OTP 26 upgrade and the way map keys are now sorted.

The problem

The OTP 26 relase notes detail the changes to how map keys are sorted:

Some map operations have been optimized by changing the internal sort order of atom keys. This changes the (undocumented) order of how atom keys in small maps are printed and returned by maps:to_list/1 and maps:next/1. The new order is unpredictable and may change between different invocations of the Erlang VM.

The folks at Fly also blogged about this in “Taking control of Map Sort Order in Elixir” and show you how to handle this in IEx to get consistent sorting.

The specific problem I experienced was a result of the cloudfront-signer library we use to Sign Cloudfront URLs for the Videos we serve. In that it builds a policy document as an Elixir map and then encodes that as JSON using poison:

defp aws_policy(resource, expiry) do
  %{
    Statement: [
      %{
        Resource: resource,
        Condition: %{
          DateLessThan: %{
            "AWS:EpochTime": expiry
          }
        }
      }
    ]
  }
end

This seems like a reasonable approach and is probably what I would have done, and has worked fine for the last 5 years or so!

However, here’s an example of the output on OTP 26:

%{
  Statement: [
    %{
      Resource: "https://example.com/",
      Condition: %{DateLessThan: %{"AWS:EpochTime": 1_692_280_023}}
    }
  ]
}
|> Poison.encode!()

"{\"Statement\":[{\"Condition\":{\"DateLessThan\":{\"AWS:EpochTime\":1692280023}},\"Resource\":\"https://example.com/\"}]}"

The issue here is that the Condition key in the JSON output comes before the Resource now which results in an invalid signature.

The solution

Thankfully the fix was easy once we understood the problem. Switching the JSON library from poison to jason produces the correct output:

%{
  Statement: [
    %{
      Resource: "https://example.com/",
      Condition: %{DateLessThan: %{"AWS:EpochTime": 1_692_280_023}}
    }
  ]
}
|> Jason.encode!()

"{\"Statement\":[{\"Resource\":\"https://example.com/\",\"Condition\":{\"DateLessThan\":{\"AWS:EpochTime\":1692280023}}}]}"