Episode 009: Channeling Re-search

Nate can't decide what to watch on Twitter, and the app restarts are killing him.

• The Twitterthon continues. "It's an infinite stream of Twitter."
• Nate wants to watch #clojurescript too.
• Just change the search string to "#clojure OR #clojurescript"?
• Should avoid hardcoding the value so we don't have to recompile every time.
• Command line argument still requires a restart.
• Let's use a curl UI (like Ep 004)!
• Wait, what?! Why curl?
  • Can send something into the running process
  • Can separate out the client
  • Builds on top of HTTP, so there are lots of tools for testing (like curl itself)
• Use a URL query string: http://localhost:8000/search?The+search+string
• "It's a true query string in the truest sense of the term 'query.'"
• "It is actually using the thing for the thing it was meant to be."
• How do we get the query string from the webserver thread to the polling loop thread?
• "This is a perfect case for a global." Oh wait, that's mutation.
• How should we structure the app? The main function must:
  • start the thread for the web server
  • start the thread for the polling loop
• Specific problem: the /search route handler function needs a way to send the new query string to the polling loop.
• With mutation: share an atom and give the handler and the polling loop a reference to it.
• No mutation? Use a core.async channel.
• A channel allows a thread to pass data to another thread.
• With no channel buffer, it will synchronize two threads ("rendezvous").
• (We lie. There is no peek function.)
• Problem: polling thread is stuck waiting on the channel, so it stops polling.
• Solution: Use alt!! to simultaneously listen to the "new search" channel and a timeout channel.
• What is a timeout channel? A channel that closes after n msecs have passed.
• New problem: the cache (for de-duplicating) has stale content after changing the query.
• Solution: same logic that adopts the new search term will reset the cache. (See Ep 007.)
• Polling loop structure:
  1. fetch
  2. process results
  3. block while listening to the channels
  4. if new search string, recur with new cache and new search string
  5. otherwise, recur with same cache and search string
• Only want the fetch in one part of the loop.
• Don't even need curl. Just type in the URL for the backend on your phone.
• core.async lets threads coordinate without having to know about each other!

Clojure in this episode:

• atom
• loop, recur
• core.async/
  • chan
  • <!, <<!, >!, >>!, put!
  • alt!, alts!
  • timeout

Related projects:

• aleph and http-kit
• core.async
• compojure