Ep 031: Eager Abstraction

Play Episode

Nate finds that trouble comes in pairs.

Related episodes:

Clojure in this episode:

Code sample from this episode:

(ns devops.week-03
  (:require
    [devops.week-01 :refer [parse-line]]
    [devops.week-02 :refer [process-log]]
    ))

(defn parse-sprinkle-error
  [line-pairs]
  (let [[first-line second-line] line-pairs
        [_whole donut-id] (some->> first-line :log/message (re-matches #"failed to add sprinkle to donut (\d+)"))
        [_whole error] (some->> second-line :log/message (re-matches #"sprinkle fail reason: (.*)"))]
    (when (and donut-id error)
      (merge first-line
             {:kind :sprinkle
              :sprinkle/donut-id donut-id
              :sprinkle/error error}))))

(defn sprinkle-errors
  [lines]
  (->> lines
       (partition 2 1)
       (map parse-sprinkle-error)
       (remove nil?)))

(defn sprinkle-errors-by-type
  [errors]
  (->> errors
       (map :sprinkle/error)
       (frequencies)))


(comment
  (process-log "sample.log" #(->> % (map parse-line) sprinkle-errors sprinkle-errors-by-type))
  )

Log file sample:

2019-05-14 16:48:55 | process-Poster | INFO  | com.donutgram.poster | failed to add sprinkle to donut 23948
2019-05-14 16:48:55 | process-Poster | INFO  | com.donutgram.poster | sprinkle fail reason: should never happen
2019-05-14 16:48:55 | process-Poster | INFO  | com.donutgram.poster | failed to add sprinkle to donut 94238
2019-05-14 16:48:55 | process-Poster | INFO  | com.donutgram.poster | sprinkle fail reason: timeout exceeded threshold
2019-05-14 16:48:55 | process-Poster | INFO  | com.donutgram.poster | failed to add sprinkle to donut 24839
2019-05-14 16:48:55 | process-Poster | INFO  | com.donutgram.poster | sprinkle fail reason: too many requests
2019-05-14 16:48:55 | process-Poster | INFO  | com.donutgram.poster | failed to add sprinkle to donut 19238
2019-05-14 16:48:55 | process-Poster | INFO  | com.donutgram.poster | sprinkle fail reason: should never happen
2019-05-14 16:48:55 | process-Poster | INFO  | com.donutgram.poster | failed to add sprinkle to donut 50493
2019-05-14 16:48:55 | process-Poster | INFO  | com.donutgram.poster | sprinkle fail reason: unknown state