Jekyll2019-07-20T15:22:24+00:00http://mitchgordon.me/feed.xmlMitchell A. GordonIntelligence, math, computing, and enough philosophy to stop doing philosophyEmpiricism in Math and AI2019-07-18T00:00:00+00:002019-07-18T00:00:00+00:00http://mitchgordon.me/philosophy/2019/07/18/empiricism-in-math-and-ai<h2 id="empricism">Empricism</h2>
<blockquote>
<p>The senses, although they are necessary for all our actual knowledge, are not
sufficient to give us the whole of it, since the senses never give anything
but instances… From which it appears that necessary truths, such as we find
in pure mathematics, and particularly in arithmetic and geometry, must have
principles whose proof does not depend on instances, nor consequently on the
testimony of the senses, although without the senses it would never have
occurred to us to think of
them…<br />
- Leibniz: Philosophical Writings</p>
</blockquote>
<p>When trying to build AI, eventually you run into this choice between empiricism
and rationalism. Where does knowledge come from? What is logic? How do we learn?</p>
<p>To me, this is a practical question rather than a philosophical one. How do I
build a machine that learns and reasons like a human?</p>
<p>To me, accepting <a href="https://en.wikipedia.org/wiki/Rationalism">rationalism</a> (like
Leibniz, above) is a non-starter for general AI because it concedes that human
thought is extra-ordinary. And if it’s extra-ordinary, then we can’t build a
machine to simulate it out of ordinary stuff like silicon.</p>
<p>And so thought can’t be abstract. Thought has to be physical, or at least
empirically observable. Thought has to follow from physical materials
interacting with the same rules as other physical systems. That assumption
lets us pursue the construction of a general AI, within the framework of
physics. (A single theory of everything.)</p>
<p>To most people, this means thought == brain, and they’re fine with that.
Thought is just neurons and biochemical reactions, governed by the same
electromagnetism etc. that exists outside of the body.</p>
<p>But if (I assume) thought is physical, then (I have to assume) everything else
is physical. Specifically, mathematics and logic. Again, this is because to
build general AI, I have to understand how to implement these things in my
machine. They have to be empirically measurable; they’re not allowed to be
hand-wavy and abstract.</p>
<h2 id="math">Math</h2>
<blockquote>
<p>All our knowledge begins with the senses, proceeds then to the understanding,
and ends with reason. There is nothing higher than reason.<br />
- Immanuel Kant, <a href="https://en.wikipedia.org/wiki/Critique_of_Pure_Reason">Critique of Pure Reason</a></p>
</blockquote>
<p>But how do you define math and logic
<a href="https://en.wikipedia.org/wiki/Operational_definition">operationally</a>, in
terms of base senses? After all, the point is that “math is true.” People
believe in math the same way they believe in God: it exists on some
higher-plane of existence, and it’s authoritative. You start with
unquestionable axioms and proceed via unquestionable logic. Adding empricial
observations is just corrupting the pureness of it.</p>
<p>Now, I’m <em>not</em> saying that “math isn’t true.” There are certainly regularities
in the universe: gravity, relativity, electromagnetism. Linear systems exist
and are predictable. Arithmetic is useful in predicting outcomes of practical
daily situations.</p>
<p>The caveat is that discovering and codifying these regularities is carried out
by humans, who are messy and error-prone. And that’s what I mean by an
empirical approach to math: viewing the people as the physical system under
study.</p>
<blockquote>
<p>A mathematician’s work is mostly a tangle of guesswork, analogy, wishful
thinking and frustration, and proof, far from being the core of discovery, is
more often than not a way of making sure that our minds are not playing tricks.<br />
- Gian-Carlo Rota</p>
</blockquote>
<p>So I view math as a natural process, involving humans (or constructed robots),
which resembles a distributed consensus algorithm. Many people <a href="http://empslocal.ex.ac.uk/people/staff/mrwatkin//zeta/RHproofs.htm">try
math</a> and
publish proofs. Only the ones that are accepted by the majority pass the
filter to become “accepted math.” This lets us carry on a single line of
reasoning over the millenia.</p>
<p>This is the essence of math: we establish a common language (axioms), we
reason about those axioms, and then we build trust in that reasoning via
distributed proof-checking.</p>
<h2 id="metaphor">Metaphor</h2>
<blockquote>
<p>The sciences do not try to explain, they hardly even try to interpret, they
mainly make models. By a model is meant a mathematical construct which, with the
addition of certain verbal interpretations, describes observed phenomena. The
justification of such a mathematical construct is solely and precisely that it
is expected to work.</p>
<p>- John Von Neumann</p>
</blockquote>
<p>But how should my “thinking machine” construct axioms or apply proven theorems
to the real world? What is the observable mechanism behind this?</p>
<p>I think about “metaphor” as the building block for this operation.
Specifically, metaphor as described by <a href="https://www.amazon.com/Metaphors-We-Live-George-Lakoff/dp/0226468011">Lakoff and
Johnson</a>,
which I’ve talked about <a href="/philosophy/2019/07/18/empiricism-in-math-and-ai.html">before</a>.</p>
<p>In <a href="http://theliterarylink.com/metaphors.html">their model</a>, everything starts
with basic senses. And as we live, we learn to associate certain experiences
with other past experiences. Argument is war, time is money, 3 gold coins is a sheep.
That’s metaphor. And they stack to build more complicated metaphors.</p>
<p>And to me, the reason we developed this faculty was so we can <a href="https://www.amazon.com/Intelligence-Understanding-Creation-Intelligent-Machines/dp/0805078533">predict the
future</a>
better. “He sank like a lead weight.” You’ve never seen him sink before, but
you’ve seen a lead weight sink. It’s like that.</p>
<p>A reasonable physical implementation of this in my machine looks like a neural
network. Of course, the details are fuzzy, but that’s what the whole field of
ML is about. How does one look at an image and decide if it looks like a dog
or a cat? In other words, how does a machine build abstract metaphor?</p>
<p>Getting back to math (and I’ve made this point <a href="/philosophy/2019/07/18/empiricism-in-math-and-ai.html">before</a>): the whole endeavor can be thought
of as building metahpors between real world-scenarios and axiomatic symbols on
the page. The symbols ought to change via rules that maintain the
metaphor. Then you can use the theorems to make predictions about your
real-world system.</p>
<blockquote>
<p>Mathematics is the study of analogies between analogies. All science is.
Scientists want to show that things that don’t look alike are really the same.
That is one of their innermost Freudian motivations. In fact, that is what we
mean by understanding.<br />
- Gian-Carlo Rota</p>
</blockquote>
<blockquote>
<p>Mathematics compares the most diverse phenomena and discovers the secret analogies that unite them.<br />
- Jean Baptiste Joseph Fourier</p>
</blockquote>
<p>To be useful, the system ought to relate via metaphor to a high variety of
physical systems. One way to do this is to keep your axioms really simple
(like things in a <a href="https://en.wikipedia.org/wiki/Set_theory">set</a>) so that
they apply whenever someone can recognize countable “things” that may or may
not be in a collection. Another way is to discover symbols that helps you
predict something about literally everything (E=mc^2).</p>
<p>But again, the application of these axioms in my “thinking machine” depends on
some physical neural networks, which have been trained through experience. The
metaphors employed during reasoning also depend on experience. I suspect
people are not different. Math doesn’t happen in a vacuum. No matter how
“pure” and “right” you think math is, eventually you need a messy brain to
pattern-match real-world systems to axioms, and back again.</p>
<p>Except for pure math, I suppose. And I can’t entirely disregard it, because even
negative numbers and complex numbers were “pure math” at some point. <a href="https://www.amazon.com/Arithmetic-Paul-Lockhart/dp/0674972236">Paul
Lockhart</a> would
say they were “extended” via symmetry and then only later reified to concrete
domains (like debts and electrical circuits). It’s curious that our aesthetic
sense of symmetry should have any relation at all to what happens in the
world. There’s probably deeper truth here, but I can’t put my finger on it.</p>
<h2 id="conclusion">Conclusion</h2>
<p>I’m an empiricist. I’ll likely live and die and empiricist. Feel free to put
“empricist” on my tombstone.</p>
<p>And this has changed how I think about math. The whole thing, to me, is
pattern-matching via metaphor which exploits regularities in the universe. Math
is only possible because for some reason, these symbols <script type="math/tex">-at^2+v_0t</script>, when
shuffled around correctly, behave exactly like a ball falling through the air.</p>
<p>It’s wild stuff.</p>
<p>And when viewed as less-than perfect art, the short-comings of math become
apparent. Some math is wrong. (Probably not linear algebra, though.) Some math
is beautiful (in the eyes of the community) but not useful. (It does not relate
via metaphor to any real-world phenomenon which we can make predictions about.)</p>
<p>There is not one “right” way to do math. There are many symbols to choose and
many ways to prove a theorem. The important thing is that the metaphor is
maintained. The symbols must refelect some regularity in the universe. And it
has to be beautiful.</p>
<blockquote>
<p>Therefore psychologically we must keep all the theories in our heads, and
every theoretical physicist who is any good knows six or seven different
theoretical representations for exactly the same physics.<br />
- Richard Feynman</p>
</blockquote>
<p>And in the end, the kind of math we create and will accept as beatiful and true
is limited by the physical constraints of our wetware. There might be
regularities in the universe that are universal but too complicated for us to
store in our brains.</p>
<p>However, all this requires abandoning the conceit that humans are special. If
you are willing to propose that you and I are fundamentally different from the
robot I am building out of stones and such (or even the people whose heads we’ve
opened up and looked inside), feel free to ignore everything I’ve said.</p>
<p>- Mitchell</p>EmpricismThe Metaphors of Arithmetic2019-06-25T00:00:00+00:002019-06-25T00:00:00+00:00http://mitchgordon.me/math/2019/06/25/metaphors-of-arithmetic<!-- Load the Paper.js library -->
<script src="/assets/js/acorn.js"></script>
<script type="text/javascript" src="/assets/js/paper-full.min.js"></script>
<!-- Define inlined PaperScript associate it with myCanvas -->
<script type="text/paperscript" canvas="myCanvas">
var center = view.center;
var MARGIN = 20;
var left = MARGIN;
var right = view.size.width - MARGIN;
function tick(x, height) {
var tick = new Path([x, center.y-height], [x, center.y+height]);
tick.strokeColor = 'black';
return tick;
}
function showText(content, pos, color='black') {
var text = new PointText(pos);
text.justification = 'center';
text.fillColor = color;
text.content = content;
text.fontFamily = 'CMSY10'
return text;
}
var group;
function divide(num) {
if (group) {group.remove()}
group = new Group();
group.addChild(showText(num, [center.x, view.size.height - MARGIN]))
var height = 5;
group.addChild(tick(left, height));
group.addChild(tick(right, height));
for (i = 1; i < num; i=i+1) {
if (num % i == 0) {
var parts = num / i;
var width = (right - left) / parts;
for (part = 1; part < parts; part=part+1) {
group.addChild(tick(left + part * width, height))
}
height += 10;
}
}
}
var parts = 1;
setInterval(function(){
if (parts < 300) {
divide(parts); parts = parts + 1;
}
}, 1000)
</script>
<div style="text-align: center">
<canvas id="myCanvas" style="width:100%"></canvas>
<h6 style="line-height: initial">Patterns In Fractions</h6>
<h6 style="margin: 0px; line-height: initial">The width is one unit. It is subdivided equally into a number of parts shown below the line.</h6>
<h6 style="line-height: initial">Groups of dividers are made taller if they also divide the width into an equal number of parts. The smaller the group, the taller the line.</h6>
</div>
<p>I’ve been reading
<a href="https://www.amazon.com/dp/0674972236/ref=cm_sw_em_r_mt_dp_U_0fOeDbT294445">Arithmetic</a>
by Paul Lockhart. It’s an excellent book, and I highly recommend it to anyone
who’s even vaguely interested in mathematics. It’s about the history of probably
the first and most fundamental mathematical abstraction: counting. The
abstractions and metaphors used to count and compare things efficiently are
simply a delight to learn about. The book is also filled with fun and crazy
questions, like the following:</p>
<blockquote>
<p>The multiples of <script type="math/tex">\wedge</script>, <script type="math/tex">\Delta</script>, and <script type="math/tex">II</script> have particularly nice
patterns in the Banana system. What are these patterns? How do they compare
with the multiples of 5, 9, and 11 in the Hindu-Arabic system?</p>
</blockquote>
<p>Besides the history, fun facts, and puzzles, I think this book is really
important for a couple other reasons…</p>
<h2 id="keeping-the-inutition">Keeping the Inutition</h2>
<p>First, I’ve noticed that while doing proofs for upper-level math classes, I
breeze through the algebra without really thinking about what it is I’m doing. I
don’t have any intuition. For example, I can write down:<sup id="fnref:1"><a href="#fn:1" class="footnote">1</a></sup></p>
<div style="text-align: center">
$$\frac{e^{a_i}\Sigma-e^{a_j}e^{a_i}}{\Sigma^2}=\frac{e^{a_i}}{\Sigma}\frac{\Sigma - e^{a_j}}{\Sigma}$$
</div>
<p>But it’s just symbol munging; I’ve basically memorized that these symbols behave
this way. If I have any intuition about what these quantities <em>mean</em> and the
metaphors at play, I’ve definitely lost it after doing any mildly complex
algebra. And like I’ve <a href="http://localhost:4000/math/2019/05/09/the-importance-of-not-taking-math-too-seriously.html">said
before</a>,
intuition and metaphor is not optional in mathematics. It’s the difference
between being a mindless mechanized robot and a human being who can see the
beauty in maths and come up with new solutions.</p>
<p>The fault lies, I think, with my grade school education. Instead of starting
with the metaphors, they teach arithmetic via <a href="https://www.ducksters.com/kidsmath/long_multiplication.php">memorizing
algorithms</a>. Which
is great if you’re a cog in the British empire’s <a href="https://www.ted.com/talks/sugata_mitra_build_a_school_in_the_cloud?language=en">bureaucratic
machine</a>:
you can still do the multiplication. You just won’t be able to ground it via
metaphor to anything that actually makes sense. <sup id="fnref:2"><a href="#fn:2" class="footnote">2</a></sup></p>
<p>So this book basically took me back to grade school and gave me a bunch of
tools and perspectives that can help me understand the rest of mathematics. I
really wish I had this book 10 years ago. After all:</p>
<blockquote>
<p>Arithmetic can be a gateway drug for mathematics. - Paul Lockhart</p>
</blockquote>
<h2 id="metaphor">Metaphor</h2>
<p>The second reason I like this book is because arithmetic is perhaps the simplest
example of using metaphor to shortcut computation, which is something I’ve been
thinking about a lot lately. This might be a stretch for some people, so let me break it down:</p>
<p>First, consider the linguistic theory of <a href="https://www.amazon.com/Metaphors-We-Live-George-Lakoff/dp/0226468011">Metaphors We Live
By</a> by
Lakoff and Johnson. Metaphor, rather than simply being a poetic device, is in
fact the elementary building block out of which our entire conceptual system
is built. And our conceptual system is basically all we have: everything we think and do can be described in terms of the metaphors we employ to relate sensory experiences to one another.</p>
<blockquote>
<p>The concepts that govern our thought are not just matters of intellect. They
also govern our everyday functioning […] what we percieve, how we get around
in the world, and how we relate to other people. Our conceptual system thus
plays a central role in defining our everyday realities.<br />
- Lakoff and Johnson</p>
</blockquote>
<p>Metaphors also stack, compounding to form more complicated and higher-level
abstractions. We don’t usually think about this system, probably because it’s
difficult to introspect and think about thinking.</p>
<p>But one way to study it is to look at how it appears in our language. ARGUMENT
is the first example given for how a concept might be metaphorical, and how it
might structure an activity. The conceptual metaphor ARGUMENT IS WAR is
reflected in everyday language in a variety of expressions:</p>
<blockquote>
<p>Your claims are <strong>indefensible</strong>.<br />
He <strong>attacked every weak point</strong> in my argument.<br />
His criticisms were <strong>right on target</strong>.<br />
I <strong>demolished</strong> his argument.<br />
I’ve never <strong>won</strong> an argument with him.<br />
You disagree? Okay, <strong>shoot!</strong><br />
If you use that <strong>strategy</strong>, he’ll <strong>wipe you out</strong>.<br />
He <strong>shot down</strong> all of my arguments.</p>
</blockquote>
<blockquote>
<p>It is important to see that we don’t just <em>talk</em> about arguments in terms of
war. We can actually win or lose arguments. We see the person we are arguing
with as an opponent. We attack his positions and we defend our own. […] Many
of the things we <em>do</em> in arguing are partially structured by the concept of
war. Though there is no physical battle, there is a verbal battle, and the
structure of an argument […] reflects this. […]</p>
</blockquote>
<blockquote>
<p><strong>The essence of metaphor is understanding and experiencing one kind of thing
in terms of another.</strong> Arguments and wars are different kinds of things […]
but ARGUMENT is partially structured, understood, and performed, and talked
about in terms of WAR. […] Moreover, this is the <strong>ordinary</strong> way of having
an argument and talking about one. - Lakoff and Johnson</p>
</blockquote>
<p>They go on to imagine a culture in which argument is instead viewed as dance,
where participants are performers, and the goal is to perform in a “balanced and
aesthetically pleasing way.” But we’re not really interested in dance or
argument; instead, we’re interested in…</p>
<h2 id="computation">Computation</h2>
<p>So here’s my hot take: <em>math</em> involves metaphor just as much as language and
everything else does. And <em>computation</em> involves a very specific kind of
metaphor: one in which the state of one side of the metaphor can be used to
<em>predict</em> specific qualities of the other side.<sup id="fnref:3"><a href="#fn:3" class="footnote">3</a></sup></p>
<blockquote>
<p>The point is that there are things, and sometimes we want to count them. - Paul Lockhart</p>
</blockquote>
<p>Consider one of the first examples of a metaphor Paul gives us: Piles of Rocks.
Suppose you’re an ancient shepherd. Not just any ancient shepherd, but a
competitive ancient shepherd: you want to know if you have a bigger flock than
that guy down the street.</p>
<p>How could you figure out who has the bigger flock? Well, if you’ve got five
sheep, and he’s got three, you can kind of just look and figure it out. This is
called number perception, and it’s built into our brains. We’re pretty good at
recognizing when things repeat (pattern recognition). The problem with number
perception is it starts to break down above 6 or 7. If you’ve both got
thirty-ish sheep, it’s really hard to tell just by looking. Also, sheep are kind
of hard to count. They run around a lot, and unless you’ve got a tower or
something, it’s hard to see all of them at once.</p>
<div style="text-align: center">
<img widht="200px" height="200px" src="http://mitchgordon.me/assets/sheep.png" />
<h6>Some sheep. <a href="https://www.agridirect.ie/blog/sheep-welfare-scheme-requirements/">Source</a></h6>
</div>
<p>Now, if it were you and me, we could probably herd all the sheep through a gate,
one by one, and count them as they go through. We do the same for our neighbor’s
flock, and then “compare” our counts somehow. There’s a problem with this
approach, however. As an ancient shepherd, you’re illiterate. You don’t even
know the names of the numbers. Maybe numbers haven’t even been invented yet.</p>
<p>So here’s where the metaphor comes in to help us out. A SHEEP IS A ROCK. Suppose
that every time we see a sheep, we put a rock in our bag. Every time we see our
neighbor’s sheep, we put a rock in his bag.</p>
<div style="text-align: center">
<img widht="200px" height="200px" src="http://mitchgordon.me/assets/rocks.jpg" />
<h6>Source - <a href="https://www.rocksforkids.com/RFK/howrocks.html">Rocks for Kids</a></h6>
</div>
<p>Now, we don’t have to compare sheep directly, but we can compare bags of rocks.
This is decidedly easier. If we’re feeling lazy, we can just see which bag feels
heavier. Or, if we want to be precise, we can line up our stones in pairs, one
from our bag and one from our neighbors, until one of us runs out. Whoever runs
out has less sheep.</p>
<p>Also, we never have to count our sheep again. (As long as we keep our bag
of rocks with us, and none of our sheep dies or gives birth, in which case we
would add or remove rocks.)</p>
<p>Notice that in order for this to work, there has to be some kind of regularity
in our world that we can exploit. This one is fairly mundane: if I have one
sheep and one rock now, I will likely still have one sheep and one rock in the
future. (If I do nothing.) That is, rocks tend to stay rocks and don’t
spontaneously break apart or burst into flames and disappear.<sup id="fnref:4"><a href="#fn:4" class="footnote">4</a></sup> Same goes for
sheep. So we can say rocks are like sheep, at least in this aspect, and we can
exploit that to make predictions about one, given the other. This is metaphor.</p>
<p>This is what I mean by short-cutting computation. I don’t have to run a full
computer simulation of my flock
<a href="https://en.wikipedia.org/wiki/The_Sims">Sims-style</a> just to know how many sheep
I’ll have at time <script type="math/tex">t</script>. I don’t even have to look at my sheep again. Instead, I
exploit some pattern I see in the world to be lazy.<sup id="fnref:5"><a href="#fn:5" class="footnote">5</a></sup></p>
<p>My conjecture is that all of math and science is basically just this, but for
more complicated relationships.</p>
<div style="text-align: center">
<img widht="200px" height="200px" src="http://mitchgordon.me/assets/coins.jpg" />
<h6>Ancient Egyptian money coins, not counting coins. - <a href="http://www.ancientegyptianfacts.com/ancient-egyptian-coins.html">Source</a></h6>
</div>
<p>Furthermore, if the whole Pile of Rocks metaphor isn’t cutting it for you (bag’s
too heavy, or rocks take too long to line up), you can stack metaphors on top of
each other to make things even more convenient.</p>
<p>Consider this metaphor: a RED ROCK IS 10 ROCKS. So if we have 10 rocks in our
bag, we can throw them out and just add a single red rock. Very convenient. Then
when we want to compare to our neighbor, we can just compare red rocks directly.
(We might need to do some exchanging to make the comparison work.) If we get too
many red rocks, we can add another metaphor: BLUE ROCK IS 10 RED ROCKS. So a blue rock is 100
rocks.</p>
<p>In fact, this is exactly how the ancient <a href="https://en.wikipedia.org/wiki/Egyptian_numerals">Egyptian
system</a> of counting worked.
They even had counting coins (similar to our colored rocks), which could be
strewn about on a table called a “counter” to do arithmetic. (And that’s why you
have a kitchen counter.)</p>
<p>And remember, all of this is based on top of the metaphor SHEEP are ROCKS,
although we could swap out sheep for anything else we want to count. The rest of
Paul’s book is basically about more metaphors, different metaphors, and ways to
stack metaphors that make counting easier. (Of course, he doesn’t exactly frame
it this way.) But all of these metaphors / different perspectives make certain
things easier to do and other things harder. (Binary makes working with powers
of 2 a breeze.)</p>
<p>So yes, it’s a good book. And it has a very clear view of how language, math,
metaphor, and computation are all inextricably intertwined. I have more to say
(and learn) about this, so I’ll probably circle back around to this topic soon.</p>
<p>- Mitchell</p>
<hr />
<div class="footnotes">
<ol>
<li id="fn:1">
<p>This is from the <a href="https://eli.thegreenplace.net/2016/the-softmax-function-and-its-derivative/">derivative of softmax</a>. <a href="#fnref:1" class="reversefootnote">↩</a></p>
</li>
<li id="fn:2">
<p>Common core actually recently started teaching the <a href="http://www.elementarymathconsultant.com/teaching-box-method-multiplication/">box
method</a>,
which moves more in the right direction, I think. <a href="#fnref:2" class="reversefootnote">↩</a></p>
</li>
<li id="fn:3">
<p>This is probably partially true of metaphors in general, but science and
engineering typically require the relationship to be consistently
predictable. (Heuristics are less valuable.) <a href="#fnref:3" class="reversefootnote">↩</a></p>
</li>
<li id="fn:4">
<p>What if rocks did tend to split in half, but only when their associated sheep gave birth? Now that would be <em>crazy</em>! (And super convenient.) <a href="#fnref:4" class="reversefootnote">↩</a></p>
</li>
<li id="fn:5">
<p>This short-cutting might not always be possible. No matter what metaphor you choose, it might not let you be lazy at all! If you’re someone like Stephen Wolfram, you might call this <a href="http://mathworld.wolfram.com/ComputationalIrreducibility.html">computational irreducibility</a> <a href="#fnref:5" class="reversefootnote">↩</a></p>
</li>
</ol>
</div>Koan of Chores2019-05-31T00:00:00+00:002019-05-31T00:00:00+00:00http://mitchgordon.me/zen/2019/05/31/koan-of-chores<p>Each morning, the monks drew lots to assign chores for the day.</p>
<p>After one drawing, a monk approached the master.</p>
<p>“Master, the lots have become marked from use, and some monks use this to select the
easiest chores. We have tried many methods, but eventually some regularity is
learned and exploited. How can we assign the chores randomly?”</p>
<p>The master replied, “what is not random?”</p>
<p>The monk thought for a moment. “When the outcome is known.”</p>
<p>The master handed the monk a blindfold.</p>
<p>“Then do not try to know.”</p>
<p>The monk was enlightened.</p>Each morning, the monks drew lots to assign chores for the day.People Used to Be Computers2019-05-10T00:00:00+00:002019-05-10T00:00:00+00:00http://mitchgordon.me/programming/2019/05/10/people-were-computers<p>Did you know people used to be computers? Yeah, back in the day, before we had
calculators or CPUs, people still needed to crunch the numbers. For example,
NASA, when they were building rockets, needed to run all kinds of numbers to
make things work. So they would actually employ people to sit down with a pencil
and paper and do the additions and multiplications necessary to solve their
physics equations. ( That’s what <a href="https://en.wikipedia.org/wiki/Hidden_Figures">Hidden
Figures</a> is about, I think. )</p>
<div style="text-align: center">
<img widht="200px" height="200px" src="http://mitchgordon.me/assets/human_computers.jpg" />
<h6>By NACA (NASA) - Dryden Flight Research Center Photo Collection - <a rel="nofollow" class="external free" href="http://www.dfrc.nasa.gov/Gallery/Photo/Places/HTML/E49-54.html">http://www.dfrc.nasa.gov/Gallery/Photo/Places/HTML/E49-54.html</a>, Public Domain, <a href="https://commons.wikimedia.org/w/index.php?curid=885426">Link</a></h6>
</div>
<p>Similarly, programming used to mean setting a schedule for, like, a play or
something. It still does, really. So programming a computer, at the time, meant
setting a schedule for some person to do a bunch of rote arithmetic (additions
and multiplications). And then when they were done, you used their answer to
build your spaceship.</p>
<div style="text-align: center">
<img widht="200px" height="200px" src="http://mitchgordon.me/assets/program.jpg" />
<h6>By <a href="//commons.wikimedia.org/wiki/User:Mindspillage" title="User:Mindspillage">Kat Walsh</a> - <span class="int-own-work" lang="en">Own work</span>, <a href="http://creativecommons.org/licenses/by-sa/3.0/" title="Creative Commons Attribution-Share Alike 3.0">CC BY-SA 3.0</a>, <a href="https://commons.wikimedia.org/w/index.php?curid=2549398">Link</a></h6>
</div>
<p>This is still true. Except now the computers aren’t people any more. They’re
robots. And when you program the robot, you basically give it a schedule of
additions and multiplications to perform. What those numbers end up meaning,
when the robot spits out the answer, is up to you.</p>
<p>I think this is the nicest way to describe what programming is to new people.
You can even keep the analogy around when you’re talking about more advanced
concepts. Consider runtime analysis: if I tell my computer (person) to solve a
problem using this program (or step by step schedule of arithmetic), will it take
them more or less time than if I give them a different program (schedule)?</p>
<p>Or try abstractions and system design: if I have a <em>really</em> long program
(schedule), it might get confusing for my computer (person) and me to keep track
of what everything means. (Why were we doing this addition on line 455 again?)
Abstraction is just figuring out how to break up that program (schedule) into
logical chunks, to make it easier for both of us to keep track of what’s going
on. And if we’re lucky, we can re-use some of those chunks, so the whole schedule
becomes shorter.</p>
<p>I could probably keep going all day, but last one: concurrency and distributed
systems. What if I hired ten people instead of one? Could I get my answer faster?
How would I have to change my program (schedule), so that everyone works together
without duplicating work? Is it even possible?</p>
<p>So yeah, next time you teach middleschoolers how to code, try this analogy. I think
it’s nice.</p>
<p>- Mitchell</p>Did you know people used to be computers? Yeah, back in the day, before we had calculators or CPUs, people still needed to crunch the numbers. For example, NASA, when they were building rockets, needed to run all kinds of numbers to make things work. So they would actually employ people to sit down with a pencil and paper and do the additions and multiplications necessary to solve their physics equations. ( That’s what Hidden Figures is about, I think. )The Importance of Not Taking Math Too Seriously2019-05-09T00:00:00+00:002019-05-09T00:00:00+00:00http://mitchgordon.me/math/2019/05/09/the-importance-of-not-taking-math-too-seriously<p>I used to take math really seriously. As a student, I worried about getting the
“right” answer or figuring out the “right” way to think about things. While
there’s nothing wrong with trying to be correct, these feelings really impeded
my ability to do good math. Anxiety makes it hard to learn new things or see
things from a different perspective, which is exactly what you need to do
good math. Instead, I started seeing math as a <em>creative</em> endeavor, and I figured
out that I do my best work when I’m having fun.</p>
<blockquote>
<p>You know, for a mathematician, he did not have enough imagination. But he has become a poet and now he is fine.<br />
- D. Hilbert</p>
</blockquote>
<p>I used to see the same thing happen when I tried to write songs or essays. If you
try to write the perfect song on your first try, you’ll never be able to write a
single word. You’ll freeze up.</p>
<p>In writing classes, a lot of people are taught to do what I call “vomitting on
paper.” You just write whatever’s on your mind, stream of conscious style. Get a
bunch of stuff down and then go back and revise. Move things around, maybe
organize stuff into bullet points. Delete some stuff. Rinse and repeat.</p>
<blockquote>
<p>There is no such thing as good writing. There is only good rewriting.<br />
- Harry Shaw, <em>Errors in English and How to Correct Them</em></p>
</blockquote>
<p>This is what I do with a lot of my blog posts. If I really want to polish, I’ll
iterate a lot. For other posts (like this one) maybe just once is enough.</p>
<p>Similarly with song writing, what I like to do is just sit down and turn on the
recorder on my phone. Always, 100% of the time. Even if I have no idea what I’m
going to play, or if I’m just fooling around. That’s because my best creative
work happens when I’m in the moment, just playing around, not worried about
sounding dumb or playing the wrong thing. And if the recorder’s on, I can always
go back and remember those moments, taking the best ones and stitching them
together into something coherent.</p>
<p>So you can kind of apply this to math, right? Most people wouldn’t
say that math is like writing an essay or a song. But really it is. And it’s
just as creative.</p>
<p>This is the problem with math homework and tests. The kids are worried about
getting bad grades, so they don’t take risks; their imagination just shuts
down. But really, to see <em>why</em> an answer might be true, you have to throw a
bunch of stuff at the wall and see what sticks. A lot of math problems can’t be
solved by just applying formulas and shuffling around symbols. You really have
to see <a href="https://betterexplained.com/articles/math-and-analogies/">the
analogies</a> and know
where you’re going, so each step you take is in the right direction.</p>
<blockquote>
<p>One should not try to prove anything that is not almost obvious. - Alexandre Grothendieck</p>
</blockquote>
<p>Really, writing a proof is just something you do <em>after</em> you’ve basically solved
the problem. It’s for the benefit of other people (and yourself), so that you
can prove to them (and yourself) that something is true. But it doesn’t really
reflect how you got there.</p>
<blockquote>
<p>We often hear that mathematics consists mainly of “proving theorems.” Is a writer’s job mainly that of “writing sentences?” - Gian-Carlo Rota</p>
</blockquote>
<p>So my process for doing math is more or less the same as writing songs and
essays. I get a copious amount of paper, write down my problem, and then I write
down anything that comes to mind, as it comes to mind. If I need to draw
pictures, I draw pictures. If I need to write down an equation and annotate what
each part really means, I do that.</p>
<p>This helps me really “go down the rabbit hole” so to speak. If I try to solve a
math problem in my head without writing anything down, I can only keep track of
1 or 2 possibilities, and follow those ideas a few steps each. But if I write
everything down, it’s like I have inifinite brain space. And then I’m not afraid
to try crazy ideas.</p>
<p>But the key thing, again, is playing around. If I ever feel like doing the math
is a chore, I know I’m going to have a bad time, and that’s the only time when math
ever really feels “hard.”</p>
<p>- Mitchell</p>I used to take math really seriously. As a student, I worried about getting the “right” answer or figuring out the “right” way to think about things. While there’s nothing wrong with trying to be correct, these feelings really impeded my ability to do good math. Anxiety makes it hard to learn new things or see things from a different perspective, which is exactly what you need to do good math. Instead, I started seeing math as a creative endeavor, and I figured out that I do my best work when I’m having fun.Koan of Will2019-04-02T00:00:00+00:002019-04-02T00:00:00+00:00http://mitchgordon.me/zen/2019/04/02/koan-of-will<p>The student saw a monk sitting in the courtyard. He did not move.</p>
<p>For forty nights and days, the student returned to the courtyard to watch the monk, who still did not move.</p>
<p>On the fortieth day, the monk died and still did not move.</p>
<p>The student asked the master, “why has he done this?”</p>
<p>“To demonstrate his free will,” the master replied.</p>
<p>“Why?”</p>
<p>“Someone told him he did not have any,” the master replied.</p>
<p>“Were they right?”</p>
<p>“It does not matter. He is dead now.”</p>The student saw a monk sitting in the courtyard. He did not move.Koan of Music2019-03-28T00:00:00+00:002019-03-28T00:00:00+00:00http://mitchgordon.me/zen/2019/03/28/koan-of-music<p>The student sought enlightenment.</p>
<p>“Master, what is best?”</p>
<p>The master answered, “play me the best music, and I will tell you the best way to live.”</p>
<p>So the student purchased a lute from a local merchant and played the master a song.</p>
<p>“That is not best,” the master said.</p>
<p>So the student traveled to the city and sought out the most famous lute players. She spent a year there, mastering seven genres.</p>
<p>She returned to the master and played her best seven songs.</p>
<p>“That is not best,” the master said.</p>
<p>In a fury, the student smashed the lute on the ground repeatedly, making a great cacophony.</p>
<p>“Ah, finally! You have played the best music… for me.”</p>
<p>The student was enlightened.</p>The student sought enlightenment.Multiplying Complex Numbers2019-03-18T00:00:00+00:002019-03-18T00:00:00+00:00http://mitchgordon.me/math/2019/03/18/multiplying-complex-numbers<!-- Load the Paper.js library -->
<script src="/assets/js/acorn.js"></script>
<script type="text/javascript" src="/assets/js/paper-full.min.js"></script>
<!-- Define inlined PaperScript associate it with myCanvas -->
<script type="text/paperscript" canvas="myCanvas">
ORIGIN = new Point(300, 150)
SCALE = 10
// (a + bi)(c + di)
// = (a + bi)c + (a + bi)di
var yaxis = new Path([ORIGIN.x, 0], [ORIGIN.x, ORIGIN.y * 2]);
var xaxis = new Path([0, ORIGIN.y], [ORIGIN.x * 2, ORIGIN.y]);
yaxis.strokeColor = 'black'
xaxis.strokeColor = 'black'
yaxis.dashArray = [4, 4]
xaxis.dashArray = [4, 4]
function convertVec(vec){
// Convert a linear algebra vector into a Paper vector
return new Point(vec.x, -vec.y) * SCALE
}
function convertPaperPos(pos) {
// Convert a paper pos to a linear algebra vector
var paperVec = pos - ORIGIN;
return new Point(paperVec.x, -paperVec.y) / SCALE
}
function showText(content, pos, color='black') {
var text = new PointText(pos);
text.justification = 'left';
text.fillColor = color;
text.content = content;
text.fontFamily = 'CMSY10'
return text;
}
function showVec(vec, {name=null, start=new Point(0,0), color='black'} = {}) {
vec = convertVec(vec);
start = ORIGIN + convertVec(start);
end = vec+start;
if (name != null) {
namePos = start + (vec / 2) + vec.normalize(15).rotate(90)
var nameText = showText(name, namePos)
}
vectorItem = new Group([
new Path([start, end]),
new Path([end, end-vec.normalize(10).rotate(30)]),
new Path([end, end-vec.normalize(10).rotate(-30)]),
nameText
]);
vectorItem.strokeColor = color;
return vectorItem
}
var mouseVec;
var a = 10; var b = 2; var c = 2; var d = 1;
var x = new Point(a, b);
var drawGroup;
function onFrame(event) {
if (drawGroup) drawGroup.remove();
y = new Point(c,d);
rightAngle = new Path();
rightAngle.moveTo(ORIGIN + convertVec((x*d).rotate(90)) + convertVec(x).rotate(90).normalize(10))
rightAngle.lineTo(ORIGIN + convertVec((x*d).rotate(90)) + convertVec(x).rotate(90).normalize(10) + convertVec(x).normalize(10))
rightAngle.lineTo(ORIGIN + convertVec((x*d).rotate(90)) + convertVec(x).normalize(10))
rightAngle.strokeColor = 'black'
drawGroup = new Group([
rightAngle,
showVec(x, {name: 'a+bi', color: 'red'}),
// showVec(x * c),
showVec((x * d).rotate(90), {name:"(a+bi)*di", color:"blue"}),
showVec(x * c, {start:(x * d).rotate(90), name: "(a+bi)*c", color: 'green'}),
showVec((x * d).rotate(90) + (x * c), {name: "(a+bi)(c+di)"}),
showText("θ", ORIGIN + convertVec((x * d).rotate(90) + (x * c)).normalize(20).rotate(20), color='purple'),
showText(`a = ${a.toFixed(2)}, b = ${b.toFixed(2)}, |a + bi| = ${x.length.toFixed(2)}`, ORIGIN + new Point(50, 50), color='red'),
showText(`c = ${c.toFixed(2)}, `, ORIGIN + new Point(50, 70), color='green'),
showText(`d = ${d.toFixed(2)}`, ORIGIN + new Point(100, 70), color='blue'),
showText(`θ = 90 - arctan(c/d) = arctan(d/c) = ${(Math.atan(d/c) / 2 / Math.PI * 360).toFixed(2)}`, ORIGIN + new Point(50, 90), color='purple'),
showText(`|(a+bi)(c+di)| = ${x.length.toFixed(2)} * ${y.length.toFixed(2)} = ${(x.length * y.length).toFixed(2)}`, ORIGIN + new Point(50, 110)),
]);
}
function onMouseMove(event) {
mouseVec = convertPaperPos(event.point)
c = mouseVec.project(x).length / x.length
d = mouseVec.project(x.rotate(90)).length / x.length
}
</script>
<p>I’ve been interested in complex numbers lately<sup id="fnref:1"><a href="#fn:1" class="footnote">1</a></sup>, but something bothered me
about complex multiplication: <strong>why is multiplying two numbers the same as adding
their angles and multiplying their lengths</strong>? Luckily, I’m
starting to figure out that <a href="https://betterexplained.com/articles/math-and-analogies/">analogies are what make math
beautiful</a>, and that
doing good math often feels like playing around.</p>
<p>So, what analogies can we use? First, complex numbers can be vectors on a plane.
Second, doing arithmetic (multiplying, adding, etc.) can be <a href="https://betterexplained.com/articles/rethinking-arithmetic-a-visual-guide/">a geometric
transformation</a>.</p>
<p>So instead of asking, “what does <script type="math/tex">(a+bi)(c+di)</script> mean?” We can ask “what
geometric transformation is the same as multiplying by <script type="math/tex">(c+di)</script>?”</p>
<p>To start, we can apply the distributive rule:</p>
<script type="math/tex; mode=display">(a+bi)(c+di) = \textcolor{green}{(a+bi)c} + \textcolor{blue}{(a+bi)di}</script>
<p>So now, instead of one transformation, we have three simpler transformations:</p>
<p><span style="color: green">1. Scale your original complex number by c.</span></p>
<p><span style="color: blue">2. Scale your original complex number by d, then
rotate it by 90 degrees.</span> <br />
(Multiplying by i is the same as rotating by 90).</p>
<p><span>3. Add those together.</span></p>
<p>When you add two vectors together, it’s the same as putting the tail of one at
the end of the other, and then drawing a vector to the result. If we plot some
examples, we can see the geometry of these transformations<sup id="fnref:2"><a href="#fn:2" class="footnote">2</a></sup>: (it should move if
you mouse over it)</p>
<div style="text-align: center">
<canvas id="myCanvas" height="300" width="600"></canvas>
</div>
<p>This trick is what
<a href="https://www.youtube.com/channel/UCYO_jab_esuFRV4b17AJtAw">3blue1brown</a> uses in
all his videos: create visual / geometric analogues for abstract concepts to
gain an intuitive and deeper understanding that connects with existing
experience. This is just one type of useful analogy, but it’s super useful, so I
want to keep practicing it.</p>
<p>Anyway, we’re interested in the black transformed vector: <script type="math/tex">(a+bi)(c+di)</script>.</p>
<p>What is the angle <script type="math/tex">\color{purple}\theta</script> between our original number and the
transformed one? From the geometry, we can see <script type="math/tex">\color{purple}\theta</script> is 90
degrees minus the angle formed by the right triangle. Luckily, we know the
length of the sides of that triangle is the same as our original complex number,
scaled by some scalars c and d:</p>
<script type="math/tex; mode=display">\textcolor{purple}{\theta} = 90 - \arctan(\frac{\textcolor{green}{c \|a+bi\|}}{\textcolor{blue}{d \|a+bi\|}}) = 90 - \arctan(\textcolor{green}{c}/\textcolor{blue}{d}) = \arctan(\textcolor{blue}{d} /\textcolor{green}{c})</script>
<p>Where <script type="math/tex">\color{red}\|a+bi\|</script> just means the length of <script type="math/tex">\color{red}(a+bi)</script>.
Notice that <script type="math/tex">\arctan(\textcolor{blue}{d} /\textcolor{green}{c})</script> is the same
as the angle of our multiplier, <script type="math/tex">(c+di)</script>. <strong>So multiplying by <script type="math/tex">(c+di)</script>
effectively rotates a complex number by the angle of <script type="math/tex">(c+di)</script>!</strong></p>
<p>If you play with the sketch above, you can see that when <script type="math/tex">d=0</script>, there’s no
rotation, when <script type="math/tex">c=0</script>, there’s a 90 degree rotation, and when <script type="math/tex">c=d</script>, the
rotation is exactly 45 degrees.</p>
<p>What about the length? Again, we can use the sides of our right triangle:</p>
<script type="math/tex; mode=display">\|(a+bi)(c+di)\| = \sqrt{\textcolor{green}{(c\|a+bi\|)^2} + \textcolor{blue}{(d\|a+bi\|)^2}} \\
= \sqrt{(c^2+d^2)\|a+bi\|^2} \\
= \sqrt{\|c+di\|^2\|a+bi\|^2} \\
= \|c+di\|\|a+bi\|</script>
<p><strong>So multiplying by <script type="math/tex">(c+di)</script> scales a number by the length of <script type="math/tex">(c+di)</script>.</strong></p>
<p>Now, we could have always figured this stuff out by applying definitions and
doing a bunch of algebra. But thinking about the geometry just makes it
feel… obvious. Note that we ignored some very important cases (what about
negative numbers?), but we can figure that out later.</p>
<p>What’s nice about having the intution is that extensions come just as naturally.
For example: finding the multiplicative inverse of a complex number. In other
words, what number undoes the transformation of multiplying by <script type="math/tex">(c+di)</script>?
Obviously it would need to rotate by the opposite angle, and then scale by the
inverse scale. More
<a href="https://betterexplained.com/articles/intuitive-arithmetic-with-complex-numbers/">here</a>.</p>
<p>So yeah, complex numbers are cool. Not sure why I never used them before, but it
seems like they would be cool <a href="https://medium.com/intuitionmachine/should-deep-learning-use-complex-numbers-edbd3aac3fb8">to use in machine
learning.</a>
What’s even cooler is that all of this complex business naturally derives from a
simple question: “how can we solve a polynomial with no real solutions?” When
you answer it, you get all this crazy geometry that’s closely related to
oscillations and trigonometry. Wild.</p>
<p>- Mitchell</p>
<div class="footnotes">
<ol>
<li id="fn:1">
<p>If you’re also interested, I highly recommend checking out BetterExplained’s <a href="https://betterexplained.com/articles/a-visual-intuitive-guide-to-imaginary-numbers/">visual intuitive guide to complex numbers</a>. If you’re really interested, check out the whole series! They go over the details of complex arithmetic, as well as euler’s formula and the Fourier Transform. Finally, you can check out 3blue1brown’s video on <a href="https://www.youtube.com/watch?v=mvmuCPvRoWQ">euler’s formula with introductory group theory</a> and <a href="https://www.youtube.com/watch?v=spUNpyF58BY">the Fourier Transform</a> for some of that and more (god I love Grant). <a href="#fnref:1" class="reversefootnote">↩</a></p>
</li>
<li id="fn:2">
<p>Multiplication is associative, so the picture we drew could also (c+di) as the original number and (a+bi) as the transformation. That would create a symmetric picture around our black product vector. Neat. <a href="#fnref:2" class="reversefootnote">↩</a></p>
</li>
</ol>
</div>Koan of Truth2019-03-13T00:00:00+00:002019-03-13T00:00:00+00:00http://mitchgordon.me/zen/2019/03/13/koan-of-truth<p>The student asked the master, “what is truth?”</p>
<p>The master grabbed his pet dog from the floor and presented it to the student.</p>
<p>“This is Truth.”</p>
<p>“That is not what I meant,” said the student.</p>
<p>“But that is what you asked,” said the master.</p>The student asked the master, “what is truth?”Koan of Rock2019-03-10T00:00:00+00:002019-03-10T00:00:00+00:00http://mitchgordon.me/zen/2019/03/10/koan-of-rock<p>The master took his student to a field, where there stood a giant rock.</p>
<p>“Move this rock,” he said.</p>
<p>The student took a step to the left.</p>
<p>The master was pleased.</p>The master took his student to a field, where there stood a giant rock.