Throughput

I found this festering in my draft blog file from back in 2015, brushed it up, and hit publish. It's a long one, top up your coffee or tipple of choice. If you want something shorter and you missed it, check out my February image of the month.

It used to be that IQ tests were all in the vogue. I took several in school, and we were assured  they were really accurate because they were computer marked. That is, as long as we coloured in the little ovals neatly with the pencils provided. I did more testing trying to figure out what I should do when I grow up, and I still wonder about that. It would be good to be ready. Any year now I figure.

For what it's worth I seemed to do well on the tests, but that just seemed to mean that I was good at the things they were testing. Or taking IQ tests, whatever. Later I read material essentially debunking the whole idea of IQ tests, testing, and that 100 was normal.

There was a job I applied for once. The first interview was with people assessing my technical ability to do the job. They loved me. The next was with the senior vice president. They were all about fitting in with the other people there. I guess he couldn't make up his mind. The next interview was with a full blown psychologist the company had on retainer. (What does that tell you about the people working there, I wonder?) I was more nervous going into that interview than any other before or since.

Let's just say we did a lot of verbal fencing. I quite enjoyed it, but I'm certain I talked myself out of a job. On most of my jobs I've been the guy that had to find a solution for the problem. There typically haven't been a lot of people around doing what I do that I could go ask. She was on about when I sought out other people, and how well I collaborated. That was the deal killer, I'm pretty sure. Several of my bosses have told me that I was regarded as a bit of a loose cannon. Effective if pointed in the right direction, but prone to rolling around in what they thought was an unpredictable way.

The other issue was when she came right out and asked me how smart I thought I was. We fenced about that for a little while too, but eventually I produced my stock answer, "just a little smarter than the average bear." She didn't like that either.

But the more I've thought about it, the more I realize there are limits on what our brains can do. In some ways we can reach them shockingly quickly. Texting and driving is a very recent example.

Let me back track a bit to simpler times. It wasn't so long ago that information travelled no more swiftly than a sailing ship or a man on horseback. Machinery was simple and obvious. Most people didn't travel much. People had the time to understand what was going on and digest it. The world seemed to be a stable place where not much changed.

Nobody really thought about how we thought, other than a few philosophers. A person was either told what to think or do, or they figured it out for themselves and got on with it. Mistakes could be fatal very quickly. Figuring out something new was rare and unusual. Since then, things have been speeding up and getting more complex.

It turns out that vision doesn't work like we thought. The idea used to be that the eye would gather all the light, an image would be produced on the retina, and the brain would understand the image. We now know the eye only receives a small fraction of the electromagnetic spectrum, and clever experiments have proved that our eyes only focus on a really small area. It's about the size of your fingernail at arms length. It's just that our focus moves around, and our brain fills in the rest.

Usually that filling in is pretty reliable, if nothing much changes. But when we focus, there's a lot we don't see. Have you taken the basketball passing test? Try it here, and see what you think. It's not even two minutes, and I'll be here when you get back.

Welcome back. Were you surprised?

Now imagine taking that test while someone was talking to you. Not just chatter about the weather you could ignore, but say, directions to some place you've never been and want to go. If you're polite you stop the video and listen to the directions. Or maybe you try to do both and get annoyed.

Just as you can only force so much liquid through a pipe of a certain size, you can only put through so much information through a human brain. Face to face conversation takes up a lot of capacity because it used to be important to our survival. It's more than processing the words, it's also the expression or emotion in the voice, and what their face and body are doing, all compared to the situation around you at the time.

This is why people thought it was no big deal to talk on cell phones and drive. People have been talking and driving for a long human lifetime now. What most people don't realize is that when two adults are driving somewhere, the talk stops periodically, and they are not aware of it. The talk stops whenever the driving situation needs more attention, such as intersections,  passing another vehicle, being near an emergency vehicle, or many other situations. This is why driving with kids can be so frustrating, when they don't shut up.

Driving takes a lot of processing power too. We have to keep our eyes in motion, constantly shifting our focus from what's in front of us, to all around us, the speedometer, the mirrors, to say nothing of the many signs along the way, stoplights and other traffic control devices, and lets not forget the girl in the red dress. Our brains have to shift along, making hundreds of calculations, how fast, rate of change, distance, timing of the lights, condition of the surface, weather, and many other factors. We know that things can change very quickly, and it could be anything from the cyclist ahead taking a spill into our path, to a vehicle suddenly crossing the center line. Some people can't cope, and know they shouldn't drive. Some are less sensible and drive anyway. Don't get me started on drunks.

Now when we add in a mobile device what happens? The person on the other end has no idea what's going on, so they keep talking. Probably too fast, and with too much detailed information. The driver probably can't hear as well as they'd like, and they feel compelled to try to pay attention to the person talking. Or worse, they're thinking of what they want to say, which is more typical conversational behavior. Their eyes might be facing forward, but the images are not being processed.

Meanwhile the world of driving is happening. Something changed that produced light that impinged on the driver's retina, but it wasn't processed because the driver was thinking about something else, and the results are not good. Ask a traffic cop how many times they've heard "But I never saw them!" and it's quite true. Oops.

There is some discussion about age and driving. Everybody knows the stories about an aged driver wreaking carnage on the roads. Advocates of the elderly driving say the statistics say that the elderly are the safest people on the road. And within certain limitations, they might well be. The problem is that we live in the real world, and shit happens. If driving under the best circumstances is cognitively challenging someone, what happens when the information processing required goes up dramatically? When there is no slack in that rope we get another oops.

I am sad to say that I no longer work with the boss that understood me and my complicated spreadsheets the best. Here is a chunk of an actual conversation in response to her question of "How do the Maxitrak, Roughneck, and Amics databases compare to each other for vessel count and data quality?" that she asked earlier in the week. (Take a deep breath.)

My response was "I started with all the A numbered vessels in Maxitrak jur_num and jur_num2 and subtracted out the ones that are not on the current ABSA inventory, and compared that to the same vessels in Roughneck that match ABSA, and discovered that w many (I no longer recall the number) match, and that x are in Maxitrak but not in Roughneck, and surprisingly, there are z many in Roughneck that are not in Maxitrak. I took that set and went back to Maxitrak to look for them. I found some have been sold, scrapped or disposed of, and that there are these enumerated errors in Roughneck. Then I compared those sets of data to Amics, after normalizing that data and excluding everything that does not have an A number. Here's the Venn diagram showing the complete comparison results between the 3 databases, and associated corrective work lists. Since A numbered vessels are the most reliable comparison, here's another Venn diagram with my estimate of the overall comparisons. Essentially anyone that tries telling you that Maxitrak is not the source of truth doesn't know what they are talking about. Not that there aren't problems with that data too."

That was all said at my conversational speed. It had taken dozens of queries to nail it down. I had an xl sheet with many tabs of data for backup. Yet she could also hold these sets of things in her head, understand what I had done to get the working sets, and compare them. The conversation got more technical after that, with her pointing to more data sources for me to look at, and wanting a couple of more detailed numbers for her to take to the steering committee. I'd also predicted she would ask a couple more questions, and I had equally detailed numbers and the explanation of exactly what they were and how I got them. She was very happy.

This is an example of a highly specific kind of throughput. Most people's eyes glaze over when I do this. Maybe you skipped that whole paragraph. Just so you know, that paragraph earned me a well north of a quarter million dollars. It convinced them I knew exactly what I was talking about and could do what they wanted done. I've only met a few other people that can do it, understanding what the end result needs to be, and being able to write the queries well enough to build the data sets, and do the manipulation to get the desired results, and be able to document the steps so people believe the numbers.

There are related kinds of work I can also do well. I can listen to someone tell the story of how they do some particular task and how it fits in with other processes. I like drawing a process flow diagram to document it, and then think about ways to improve it, or to adapt it for a new computer system.

For both those tasks I'm sometimes staring at a whiteboard covered with little boxes with arrows between them, and cryptic notes. Like this.

Once I'm dived in, I do not want to be disturbed. I'm holding a bunch of stuff in my head all at once, and it's easy to lose it. There are lots of other jobs that have the same sort of intellectual requirements. Software development is one I'm familiar with, though I can't do it.

Some people say they can multi-task while doing such jobs, and they're lying. Science says so. What they are actually trying to do is switch their attention between two tasks quickly, and hold both in their head.

If the tasks are simple, like playing tick tack toe with child while talking on the phone to a buddy about nothing in particular, you're fine. Now take another look at that diagram, and imagine someone asking about a similar, equally complicated diagram relating to something else. Once you start down the path of answering that distracting question your brain dumps the data relating to the first diagram, and you start thinking about the other one. You have to get back into it again, even if you dealt with it quite recently. And then, you gave to get back into this one again. Usually I'll take a break and look at email, or get a drink of water or do something else for a moment while I'm distracted.

There is a significant effort to switch your attention, and the effort it takes to do that is taken from the task, or tasks at hand. This is one reason why it can be energizing to do a deep dive into a problem and work through it, and why people are often so tired at the end of the day. They spent all their energy and attention switching between tasks, rather than on a task.

It isn't just work. Think about watching a movie. For most of us this is entertainment, but I don't enjoy them as much as I used to. Yes, Hollywood is making stupider movies now, but that's another blog. Movies have changed. It's all frantic, often meaningless action now, but even worse are the cuts. The average shot length has decreased over the years to less than 5 seconds now. Just like switching attention between tasks, it takes mental effort to track the movie through the cut.

Now, (making the leap to today, keep up) we have Trump telling us all sorts of nonsense. He tells the lies so thick and fast most people can't keep up anymore, especially if they've been fed similar lies already. People get all distracted by the flash and dazzle and the sheer quantity, and don't stop to think any of them through. Even if they do, Trump is now 6 lies down the road, burying what he doesn't want you to see under a pile of bullshit. Russia, folks, stay focussed on Russia, and his team's dealings with it. If it had been Clinton even suspected of doing half that stuff, the Republican's would be orbital with indignation.

Part of the secret to happiness is to stay focussed on one thing at a time. Preferably the most important and urgent task facing you, especially if you're driving. After that it doesn't matter if it's getting that spreadsheet back to the boss, or tweaking camera settings to get that image just right to take it from nice to wow. Enjoy doing it. Then when you're done move onto the next task. Don't get lost in the jumble of buildings and tasks and priorities. Pick the one you like, then focus in.