Ep 023: Poster Child
Nate gets messy finding ingredients for his algorithm cake.
- Last week we focused on how to determine what to post.
- This week we focus on getting the data so we can compare it.
- (01:55) Once again, we’ll use
componentto organize our app.
- What components should we have?
- Component 1: The worker that wakes up, checks the DB, checks Twitter, and posts as necessary.
- Debate: Do we need more than one component?
- Question: What does that worker need? Those should be components.
- (03:00) Component 2: The database connection.
- Needs to be threadsafe
- Allows all the DB logic in one place.
startmethod is a natural place to do migrations, indexing, etc.
- (04:45) Aside: quick component refresher
- Very lightweight mechanism for shared, stateful resources.
- Dependency injection: components depend on other components, but framework instantiates them.
- Each component has two lifecycle methods
- Setup state in
start: initialize internal data, open connection, prep work, etc.
- Tear it all down in
stop: finishing work, closing connections, etc.
- Declare all your components in a
- (06:00) What should we call Component 1? “Poster”?
- “It could be the poster child of components!”
- (06:50) Component 3: The Twitter API handle
- What is a “Twitter API handle” (aka the “Twitter handle”)?
- Data structure and code that handles authenticating and making requests.
- Not a long standing socket connect (like the DB), but there is still state (auth key).
- (08:50) What happens when the cached credentials expire?
- Two options from Ep 006:
- Request function returns a tuple:
[updated-handle, result]. The handle will only change when it has to re-auth.
- Use an
atomfor the handle. Request function mutates the handle when it has to re-auth.
- Request function returns a tuple:
- In both cases, the Twitter wrapper deals with re-auth. The difference is whether the code that uses the handle needs to know about it.
- We’ll use #2 for our Twitter component.
- We must still consider a race condition between components who all use an expired handle at the same time. Extra work will be done, but the result will still be correct.
- If components use separate handles, they can’t benefit from sharing the re-auth.
- (12:55) We need a way to trigger “waking up”
at-atto schedule an interval for calling a function
- Backed by Java’s
- “It’s good to have your components be tidy and clean up after themselves.”
- Important to call
component.repl/resetdoesn’t make more and more timers!
- (16:10) We have the ingredients to implement the algorithm
- Basic steps:
- Wrap it all in an exception handler
- Use a sequence in a let block to assign results back from Twitter and DB
- Take results and pass them to pure function to determine what to do next
- Complication: order dependent. We need last posted Tweet to know how far back to fetch from Twitter
- New steps:
- Fetch from DB: last posted and next scheduled
- Use last posted ID to fetch from Twitter
- Pass tweets and next scheduled tweet to pure function to determine if we should post
- If we need to post, try posting to Twitter and capture result
- If success, write tweet ID into database to mark as “completed”
- No matter what, write the result to the “attempt” log
- (22:00) Feels very imperative
- “Do I/O. Do some logic. Do I/O. Do some logic. All those little bits of logic are very hard to test.”
- OO answer: mock the resources
- Mocking makes more problems: now you have to implement all sorts of fake logic
- “You just start grabbing more and more side effects and glomming them on to this big ball of mud, just so you can test a little bit of logic.”
- “Next thing you know, you’re developing a whole vocabulary of mock creation.”
- We’ll look at this more next week.
Message Queue discussion:
- (25:00) We were mentioned on the Illegal Argument podcast
- Comparing Clojure REPL and Smalltalk REPL.
- Common problem: state in the REPL does not reflect what is in the source.
- For us, connected editor helps avoid that: we run pieces directly from our source files.
- Still can end up out of sync: dangling symbol references.
tools.namespace.repl/refreshwill find those dangling references.
- Can still build up in pieces, but can use
refreshto check it all at once.
- Twitter handle, retrying on failure
Clojure in this episode: