The Myth of Objective Tech Screens

Can you tell me about a time..

Can you tell me about a failure you had and what you learnt from it?

Sure thing!

My situation was that my startup hadn’t reached product/market fit yet, my savings were dwindling so my task was to find a tech job to get some income again. I sprung to action, preparing all the basics of computer science and machine learning and made sure I could reimplement all the algorithms I ever worked on from scratch. Unfortunately, the result was a rejection - they never told me why but I figure it was a culture fit kind of thing or maybe it’s because I didn’t know what R-CNNs were or who knows apparently they’re not allowed to tell me what went wrong.

And what did you learn from it?

I learnt that most people don’t respect risk takers that fail. They say they do because they figure they should, maybe they even view themselves as contrarian risk takers. It was real risky not taking that consulting job to get a PhD in Machine Learning from a small college in Boston. In all cases, maybe I’m a bit salty - it was painful getting critical feedback from people whose only real risk they took in life was disagreeing with their manager once in a meeting. I feel like they’re just a bunch of LinkedIn bots.

LinkedIn bots?

LinkedIn bots are people that only ever re-share corporate announcements, I’m never really sure what their opinions about anything are. Also no interesting human communicates in a STAR format.

This is fixed mentality thinking, you’re focusing too much on others. What would have happened had you adopted a growth mentality?

Well if I was actually really smart, I would have just started a really successful business. I would have made a best selling game with top AI that I would have packaged as a service, I’d have focused a lot more on getting a Twitter audience so I can broadcast my engineering work and find customers and peers in a scalable manner. I’d have trusted my gut feeling more and done what I thought was right instead of aggregating opinions from people I admire.

That’s great, so just so you know we are looking for pure software engineers but umm we’ll stay in touch!

---

Thank you for taking the time, we have about 15 min left, do you have any questions for me?

Thank you! I do. I was wondering what your favorite part of your job is

You know we are big but we are like a startup, we work on cool problems - yes yes it’s very good

---

Ok so imagine you’re designing a taxi cab system and users can make bookings

Ok so a booking has a start date and a predicted end date ok I’ll make a class for that, and um probably need a class for drivers ok.. So to see if there’s a conflict I’ll search through all the bookings for each driver so I need to iterate over the keys of the dictionary. I forget the syntax lemme just Google it quickly.

No

Um no? I forget if it was iter.items() or keys or something else. The syntax changed from Python 2 to 3

Oh so you don’t know Python?

---

Can you explain to me what Backpropagation is?

Sure! Backprop is really just a special case of Automatic Differentiation and my blogpost is actually the #1 hit on Google for Automatic Differentiation. We can actually skim it together if you’re interested

I’m sorry I can’t check content outside of the context of this interview, can you just explain the core ideas?

Dual numbers, primal form and dual form

Hmm, I was hoping to hear you mention the chain rule but OK let’s stay in touch

---

Can you explain to me how Logistic Regression works?

*Write down the full derivation like a speed demon*

Can you implement Breadth First Search?

*Forgot I need to use a queue, realize 15 min have gone bye, start hyperventilating, realize I’m not getting my dream job*

---

So our interview process is a bit different, you’re going to take a 4h exam on Math, CS Theory, ML and Statistics

Sounds great! *ace the interviews*

Congrats! Next is the coding screen. How would you estimate Pi numerically?

Interesting I’ve never thought about that - *think a bunch*. Oh I see I can sample random points and some will land inside and some outside and take the ratio oooh that’s a cool trick.

Hmm you were a bit slow

---

Can you tell me about a time someone gave you negative feedback?

Um sure! I had just graduated from college, I was working on a BI system for ads and was lost for months! There were so many teams and meetings and it was hard to keep track of all the acronyms. I asked for help and people told me to just be patient, take notes in meetings and eventually things would start coming together. So there I was engaging in my note taking therapy when one of my seniors chewed me out and asked me if I wanted to make decisions or take notes.

I held back a tear, my senior took me aside and started coaching on how to reduce ambiguity of complex projects by producing wrong documentation and sharing it publicly so people would correct me. I just had to get rid of my ego, I’m honestly very grateful for this feedback. It changed my life.

Hmm so you were lost for months - I see..

---

Ok so we’re going to get started with a coding screen. Can you print all the permutations of a string?

Ok so I need to do this recursively, OK so my base case is the string is empty or wait the string is full, let’s see. Ok then I have a for loop to make multiple recursive calls or no maybe I should backtrack. Ummm *fumbles for 30 min*

Well we only have 10 mins left so maybe just walk me through at a high level how you would implement a queue with only stacks?

I’ll code it up. Queue contains elements and you can enqueue and dequeue by using 2 stacks like so. *Codes up fully working solution in 2min*

Uhum, we’ll be in touch

---

Ok so what’s an activation function?

It’s a way to bound the output of a network to some range of values

What’s the best one?

I don’t think there is clear consensus, I use ReLu, Sigmoid and Tanh a bunch. They are all simple to compute and have simple derivatives. ReLU bounds values to [0, ∞], softmax [0,1] and tanh [-1,1]. ReLU is used for the vanishing gradient problem. So I don’t know, really depends on your goal.

What’s the best one?

Machine Learning is an empirical discipline I would just try them all out and see what works on my specific problem

I see. How many CNN architectures do you know?

I’ve heard of a bunch but I’ve only worked with Resnet and Unet

So you haven’t worked with Yolo?

I’m pretty sure I can figure it out, anecdotally most CNN architectures I’ve seen use CNNs, some version of normalization and some activation function that they repeat in blocks

Yes or no have you worked with Yolo?

No

---

Great to meet you Mark, OK so can you design an online Python Debugger for me?

*Spend a day working on it* This is really cool, OK so I’ll make a Flask app and then take the first submit from the user to take in their request, turn it into a Python object function, trace it so I can go through it line by line. Ok great it works!

That it does but it does seem a bit brittle

Yeah I haven’t spent too much time doing web development so some things may be funky. Most of my past experience has been in ML stuff but I really love your product and company. If anything I’m just happy I met you.

I understand let’s stay in touch on Twitter

---

Ok so imagine you have a self-driving car but it has some uncertainty around where its wheels are, how would you structure such a problem?

Hmm so I guess we can measure at various points in time where we think the wheels are keep track of those in a list so we have 4. Hmm but this list will be really big so maybe keep a fixed window. Ok so inputs for the same sensor should be correlated and even though the values are fixed we can rotate them cyclically and assume some gaussian noise, how does this look?

Nice you just implemented a Kalman filter

Oh wow, I didn’t realize that. Thank you!

My pleasure!

---

Describe a long-term project that you managed. How did you keep everything moving along in a timely manner?

Well probably that would be at my startup, I found it best to make small measurable progress and sharing it online. Although sometimes it’s difficult because it’s very easy to go down a rabbit hole working on irrelevant things. At bigger companies projects are well established with clear goals and those are easy, you just need to do the work in an organized way. What I found much harder was deciding what was important because I was splitting my time between all the disciplines.

Hmm do you have another example from a bigger company you worked at?

So what I did was I put everything in a big table with the really good stuff in green and with the late stuff in red, then for red ones I open up a ticket that repeats the title with the name of the engineer that I’m casually blaming for the delays. VPs never liked the due dates so I’d encourage engineers to conjure code instantaneously so no-one would yell at me.

---

Suppose you were a video game designer and your players have been complaining about how bad the AI is, what would you do next?

*Finally someone who understands me*

So players are complaining that the AI sucks so we can start training it with Reinforcement Learning. RL really just dumps experiences in a key value store. Simple algorithms like Q learning or DDPG are the easiest to scale and we can either do imitation learning or self play.

Ok so you learn that the battery is now running out fast, what do you do?

So most likely culprit is our RL telemetry but in all cases I’d profile it in Unity and sort processes by their memory and compute usage. But for the telemetry specifically, we can only log what we need by taking the transforms of each game object, we also need to make sure that we have good data locality so using ECS in our app is probably a good idea. We can also send telemetry data when the user isn’t using the app.

Ok now your backend is slow

Well we can prioritize different requests based on whether they’re gameplay or telemetry specific, we could even use UDP for telemetry. Would also make sense to replicate our key value store so it can be updated quickly and add a load balancer etc..

So now users are complaining that the AI is too good

We could have a bunch of AIs sorted by difficulty then binary search through it to find a good match for the player as they play more games or we could just teach the AI to imitate the player so it becomes better as the player gets better.

So what’s the loss function?

Loss(What Player did, What Bot predicted player would do)

Thank you so much I really enjoyed this

Same here

---

So what did you learn from all this?

People like to hire people that think or sound like them. There’s nothing objective about trivia, rubrics or principles.

I’m sure some things went well

I was very grateful to the interviewers who helped me show my best and I’ll try my best to extend that same courtesy to people I interview.

Acknowledgements

Thank you to Hadi Inja for prompting me to write this, the Robot Overlords Discord community for early feedback and Sarah Saroufim for finding an embarrassing amount of typos.