Chapter 2
Vectors
Yellow fever vaccine should be administered
10 to 12 days before
exposure to the vector.
— The United States Dispensatory (1978)
Vectors are the formal mathematical entities we use to do 2D and 3D math. The word vector
has two distinct but related meanings. Mathematics books, especially those on linear algebra,
tend to focus on a rather abstract definition, caring about the numbers in a vector but not
necessarily about the context or actual meaning of those numbers. Physics books, on the other
hand, tend towards an interpretation that treats a vector as a geometric entity to the extent
that they avoid any mention of the coordinates used to measure the vector, when possible. It's no
wonder that you can sometimes find people from these two disciplines correcting one another on
the finer points of “how vectors really work.” Of course the reality is that they are
both right, and to be proficient
with 3D math, we need to understand both interpretations of vectors and how the two
interpretations are related.
This chapter introduces the concept of vectors. It is divided into the following sections.

Section 2.1 covers some of the basic mathematical properties of
vectors.

Section 2.2 gives a highlevel introduction to the geometric
properties of vectors.

Section 2.3 connects the mathematical definition with the geometric
one and discusses how vectors work within the framework of Cartesian coordinates.

Section 2.4 discusses the often confusing relationship between
points and vectors and considers the rather philosophical question of why it is so
difficult to make absolute measurements.

Sections 2.5–2.12 discuss
the fundamental calculations we can perform with vectors, considering both the
algebra and geometric interpretations of each operation.

Section 2.13 presents a list of helpful vector algebra
laws.
2.1Mathematical Definition of Vector,
and Other Boring Stuff
To mathematicians, a vector is a list of numbers. Programmers will recognize the
synonymous term array. Notice that the STL template array class in C++ is named
vector, and the basic Java array container class is java.util.Vector.
So mathematically, a vector is nothing more than an array of numbers.
Yawn… If this abstract definition of a vector doesn't inspire you, don't worry. Like many
mathematical subjects, we must first introduce some terminology and notation before we can get to
the “fun stuff.”
Mathematicians distinguish between vector and scalar (pronounced “SKAYlur”) quantities.
You're already an expert on scalars—scalar is the technical term for an ordinary number. We
use this term specifically when we wish to emphasize that a particular quantity is not a vector
quantity. For example, as we will discuss shortly, “velocity” and “displacement” are vector
quantities, whereas “speed” and “distance” are scalar quantities.
The dimension of a vector tells how many numbers the vector contains. Vectors may be of
any positive dimension, including one. In fact, a scalar can be considered a 1D vector. In this
book, we primarily are interested in 2D, 3D, and (later) 4D vectors.
When writing a vector, mathematicians list the numbers surrounded by square brackets, for
example,
$[1,2,3]$
. When we write a vector inline in a paragraph, we usually put commas between
the numbers. When we write it out in an equation, the commas are often omitted. In either case, a
vector written horizontally is called a row vector. Vectors are also frequently written
vertically:
A 3D column vector
$$\left[\begin{array}{c}1\\ 2\\ 3\end{array}\right].$$
A vector written vertically is known as a column vector. This book uses both notations.
For now, the distinction between row and column vectors won't matter. However, in
Section 4.1.7 we discuss why in certain circumstances the
distinction is critical.
When we wish to refer to the individual components in a vector, we use subscript notation. In
math literature, integer indices are used to access the elements. For example
${v}_{1}$
refers to the first element in
$\mathbf{v}$
. However, we are specifically interested in 2D, 3D,
and 4D vectors rather than vectors of arbitrary dimension
$n$
, so we rarely use this notation.
Instead, we use
$x$
and
$y$
to refer to the elements in a 2D vector;
$x$
,
$y$
, and
$z$
to refer
to the elements in a 3D vector; and
$x$
,
$y$
,
$z$
, and
$w$
to refer to the elements in a 4D
vector. This notation is shown in Equation (2.1).
Vector subscript notation
$$\begin{array}{}\text{(2.1)}& \begin{array}{ccl}\mathbf{a}=\left[\begin{array}{c}1\\ 2\end{array}\right]& & \begin{array}{l}{a}_{1}={a}_{x}=1\\ {a}_{2}={a}_{y}=2\end{array}\\ \\ \mathbf{b}=\left[\begin{array}{c}3\\ 4\\ 5\end{array}\right]& & \begin{array}{l}{b}_{1}={b}_{x}=3\\ {b}_{2}={b}_{y}=4\\ {b}_{3}={b}_{z}=5\end{array}\\ \\ \mathbf{c}=\left[\begin{array}{c}6\\ 7\\ 8\\ 9\end{array}\right]& & \begin{array}{l}{c}_{1}={c}_{x}=6\\ {c}_{2}={c}_{y}=7\\ {c}_{3}={c}_{z}=8\\ {c}_{4}={c}_{w}=9\end{array}\end{array}\end{array}$$
Notice that the components of a 4D vector are not in alphabetical
order. The fourth value is
$w$
. (Hey, they ran out of letters in
the alphabet!)
Now let's talk about some important typeface conventions that are used in this book. As you
know, variables are placeholder symbols used to stand for unknown quantities. In 3D math,
we work with scalar, vector, and (later) matrix quantities. In the same way that it's important
in a C++ or Java program to specify what type of data is stored by a variable, it is important
when working with vectors to be clear what type of data is represented by a particular variable.
In this book, we use different fonts for variables of different types:

Scalar variables are represented by lowercase Roman or Greek
letters in italics:
$a$
,
$b$
,
$x$
,
$y$
,
$z$
,
$\theta $
,
$\alpha $
,
$\omega $
,
$\gamma $
.

Vector variables of any dimension are represented
by lowercase letters in boldface:
$\mathbf{a}$
,
$\mathbf{b}$
,
$\mathbf{u}$
,
$\mathbf{v}$
,
$\mathbf{q}$
,
$\mathbf{r}$
.

Matrix variables are represented using uppercase
letters in boldface:
$\mathbf{A}$
,
$\mathbf{B}$
,
$\mathbf{M}$
,
$\mathbf{R}$
.
Note that other authors may use different conventions. One common
convention used frequently when writing vectors by hand, is to draw
an arrow over the vector, for example,
$\overrightarrow{a}$
.
Before we go any further, a bit of context is in order concerning the perspective that we are
adopting about vectors. The branch of mathematics that deals primarily with vectors and matrices
is called linear algebra, a subject that assumes the abstract definition given previously:
a vector is an array of numbers. This highly generalized approach allows for the exploration of a
large set of mathematical problems. In linear algebra, vectors and matrices of dimension
$n$
are
used to solve a system of
$n$
linear equations for
$n$
unknowns, without knowing or caring what
physical significance, if any, is attached to any of the numbers. This is certainly a fascinating
and highly practical study, but it is not of primary interest to our investigation of 3D math.
For 3D math, we are mostly concerned with the geometric interpretations of vectors and vector
operations.
Our focus is geometric, so we omit many details and concepts of linear algebra that do not
further our understanding of 2D or 3D geometry. Even though we occasionally discuss properties
or operations for vectors of an arbitrary dimension
$n$
, we will usually focus on 2D, 3D, and
(later) 4D vectors and matrices. Even when the numbers in a vector do not have any
physical significance, the ability to visualize the linear algebra operations is of some utility,
so learning how to interpret the operations geometrically is useful even in nonphysical
applications. Some more context about how the topics in this book fit into the bigger picture of
linear algebra can be found in Section 4.3.
2.2Geometric Definition of Vector
Now that we have discussed what a vector is mathematically, let's look at a more geometric
interpretation of vectors. Geometrically speaking, a vector is a directed line segment that has
magnitude and direction.

The magnitude of a vector is the length of the vector. A
vector may have any nonnegative length.

The direction of a vector describes which way the vector
is pointing in space. Note that “direction” is not exactly the same as
“orientation,” a distinction we will reexamine in
Section 8.1.
Figure 2.1A 2D vector
Let's look at a vector. Figure 2.1 shows an illustration of a vector in 2D.
It looks like an arrow, right? This is the standard way to represent a vector
graphically, since the two defining characteristics of a vector are captured: its
magnitude and direction.
We sometimes refer to the head and tail of a vector. As shown in
Figure 2.2, the head is the end of the vector with the arrowhead on it (where
the vector “ends”), and the tail is the other end (where the vector “starts”).
Where is this vector? Actually, that is not an appropriate question. Vectors do not have
position, only magnitude and direction. This may sound impossible, but many quantities we deal
with on a daily basis have magnitude and direction, but no position. Consider how the two
statements below could make sense, regardless of the location where they are applied.

Displacement. “Take three steps forward.” This sentence
seems to be all about positions, but the actual quantity used in the
sentence is a relative displacement and does not have an absolute
position. This relative displacement consists of a magnitude (3
steps) and a direction (forward), so it could be represented by a
vector.

Velocity. “I am traveling northeast at 50 mph.”
This sentence describes a quantity that has magnitude (50 mph) and
direction (northeast), but no position. The concept of “northeast
at 50 mph” can be represented by a vector.
Notice that displacement and velocity are technically different from the terms
distance and speed. Displacement and velocity are vector quantities and therefore
entail a direction, whereas distance and speed are scalar quantities that do not specify a
direction. More specifically, the scalar quantity distance is the magnitude of the vector
quantity displacement, and the scalar quantity speed is the magnitude of the vector quantity
velocity.
Because vectors are used to express displacements and relative differences between things, they
can describe relative positions. (“My house is 3 blocks east of here.”) However, you should
not think of a vector as having an absolute position itself, instead, remember that it is
describing the displacement from one position to another, in this case from “here” to “my
house.” (More on relative versus absolute position in Section 2.4.1.) To help
enforce this, when you imagine a vector, picture an arrow. Remember that the length and direction
of this arrow are significant, but not the position.
Since vectors do not have a position, we can represent them on a diagram anywhere we choose,
provided that the length and direction of the vector are represented correctly. We often use
this fact to our advantage by sliding the vector around into a meaningful location on a diagram.
Now that we have the big picture about vectors from a mathematical and geometric perspective,
let's learn how to work with vectors in the Cartesian coordinate system.
2.3Specifying Vectors with Cartesian Coordinates
When we use Cartesian coordinates to describe vectors, each coordinate measures a signed
displacement in the corresponding dimension. For example, in 2D, we list the displacement
parallel to the
$x$
axis, and the displacement parallel to the
$y$
axis, as illustrated in
Figure 2.3.
Figure 2.4 shows several 2D vectors and their values. Notice that the position of each vector on the diagram is irrelevant. (The axes are conspicuously
absent to emphasize this fact, although we do assume the standard convention of
$+x$
pointing to
the right and
$+y$
pointing up.) For example, two of the vectors in
Figure 2.4 have the value
$[1.5,1]$
, but they are not in the same place on
the diagram.
3D vectors are a simple extension of 2D vectors. A 3D vector
contains three numbers, which measure the signed displacements in
the
$x$
,
$y$
, and
$z$
directions, just as you'd expect.
We are focusing on Cartesian coordinates for now, but they are not the only way to describe
vectors mathematically. Polar coordinates are also common, especially in physics
textbooks. Polar coordinates are the subject of Chapter 7.
2.3.1Vector as a Sequence of Displacements
One helpful way to think about the displacement described by a vector is to break out the vector
into its axially aligned components. When these axially aligned displacements are combined, they
cumulatively define the displacement defined by the vector as a whole.
For example, the 3D vector
$[1,3,4]$
represents a single displacement, but we can visualize this
displacement as moving 1 unit to the right, 3 units down, and then 4 units forward. (Assume our
convention that
$+x$
,
$+y$
, and +z point right, up, and forward, respectively. Also note that we
do not “turn” between steps, so “forward” is always parallel to
$+z$
.) This displacement is
illustrated in Figure 2.5.
The order in which we perform the steps is not important; we could move 4 units forward, 3 units
down, and then 1 unit to the right, and we would have displaced by the same total amount. The
different orderings correspond to different routes along the axially aligned bounding box
containing the vector. Section 2.7.2 mathematically verifies this
geometric intuition.
2.3.2The Zero Vector
For any given vector dimension, there is a special vector, known as the zero vector, that
has zeroes in every position. For example, the 3D zero vector is
$[0,0,0]$
. We denote a zero
vector of any dimension using a boldface zero:
$\mathbf{0}$
. In other words,
The zero vector
$$\mathbf{0}=\left[\begin{array}{c}0\\ 0\\ \vdots \\ 0\end{array}\right].$$
The zero vector is special because it is the only vector with a magnitude of zero. All other
vectors have a positive magnitude. The zero vector is also unique because it is the only vector
that does not have a direction.
Since the zero vector doesn't have a direction or length, we don't draw it as an arrow like we do
for other vectors. Instead, we depict the zero vector as a dot. But don't let this make you
think of the zero vector as a “point” because a vector does not define a location. Instead,
think of the zero vector as a way to express the concept of “no displacement,” much as the
scalar zero stands for the concept of “no quantity.”
Like the scalar zero you know, the zero vector of a given dimension is the additive
identity for the set of vectors of that dimension. Try to take yourself back to your algebra
class, and retrieve from the depths of your memory the concept of the additive identity: for any
set of elements, the additive identity of the set is the element
$x$
such that for all
$y$
in the
set,
$y+x=y$
. In other words, when we add the zero vector to
any other vector, we get that vector:
$\mathbf{0}+\mathbf{a}=\mathbf{a}$
.
Section 2.7 deals with vector addition.
2.4Vectors versus Points
Recall that a “point” has a location but no real size or thickness. In this chapter, we have
learned how a “vector” has magnitude and direction, but no position. So “points” and
“vectors” have different purposes, conceptually: a “point” specifies a position, and a
“vector” specifies a displacement.
But now examine
Figure 2.6, which compares an illustration
from Chapter 1 (Figure 1.8),
showing how
2D points are located, with a figure from earlier in this chapter (Figure 2.3),
showing how 2D vectors are specified. It seems that there is a strong relationship between points
and vectors. This section examines this important relationship.
2.4.1Relative Position
Section 2.2 discussed the fact that because vectors can describe
displacements, they can describe relative positions. The idea of a relative position is fairly
straightforward: the position of something is specified by describing where it is in relation to
some other, known location.
This begs the questions: Where are these “known” locations? What is an “absolute” position?
It is surprising to realize that there is no such thing! Every attempt to describe a position
requires that we describe it relative to something else. Any description of a position is
meaningful only in the context of some (typically “larger”) reference frame. Theoretically, we
could establish a reference frame encompassing everything in existence and select a point to be
the “origin” of this space, thus defining the “absolute” coordinate space. However, even if
such an absolute coordinate space were possible, it would not be practical. Luckily for us,
absolute positions in the universe aren't important. Do you know your precise position in
the universe right now? We don't know ours, either.
2.4.2The Relationship between Points and Vectors
Vectors are used to describe displacements, and therefore they can describe relative positions.
Points are used to specify positions. But we have just established in
Section 2.4.1 that any method of specifying a position must be relative.
Therefore, we must conclude that points are relative as well—they are relative to the origin of
the coordinate system used to specify their coordinates. This leads us to the relationship
between points and vectors.
Figure 2.7 illustrates how the point
$(x,y)$
is
related to the vector
$[x,y]$
, given arbitrary values for
$x$
and
$y$
.
As you can see, if we start at the origin and move by the amount
specified by the vector
$[x,y]$
, we will end up at the location
described by the point
$(x,y)$
. Another way of saying this is that
the vector
$[x,y]$
gives the displacement from the origin to the
point
$(x,y)$
.
This may seem obvious, but it is important to understand that points and vectors are conceptually
distinct, but mathematically equivalent. This confusion between “points” and “vectors” can
be a stumbling block for beginners, but it needn't be a problem for you. When you think of a
location, think of a point and visualize a dot. When you think of a displacement, think of a
vector and visualize an arrow.
In many cases, displacements are from the origin, and so the distinction between points and
vectors will be a fine one. However, we often deal with quantities that are not relative to the
origin, or any other point for that matter. In these cases, it is important to visualize these
quantities as an arrow rather than a point.
The math we develop in the following sections operates on “vectors” rather than “points.”
Keep in mind that any point can be represented as a vector from the origin.
Actually, now would be a good time to warn you that a lot of people take a much firmer stance on
this issue and would not approve of our cavalier attitude in treating vectors and points as
mathematical equals. Such hardliners will tell you, for example, that while you can add
two vectors (yielding a third vector), and you can add a vector and a point (yielding a point),
you cannot add two points together. We admit that there is some value in understanding these
distinctions in certain circumstances. However, we have found that, especially when writing code
that operates on points and vectors, adherence to these ethics results in programs that are
almost always longer and never faster. Whether it makes the code cleaner or easier to understand is a highly subjective
matter. Although this book does not use different notations for points and vectors, in general
it will be clear whether a quantity is a point or a vector. We have tried to avoid presenting
results with vectors and points mixed inappropriately, but for all the intermediate steps, we
might not have been quite as scrupulous.
2.4.3It's All Relative
Before we move on to the vector operations, let's take a brief philosophical intermission.
Spatial position is not the only aspect of our world for which we have difficulty establishing an
“absolute” reference, and so we use relative measurements. There are also temperature,
loudness, and velocity.
Temperature. One of the first attempts to make
a standard temperature scale occurred about AD 170, when Galen
proposed a standard “neutral” temperature made up of equal
quantities of boiling water and ice. On either side of this
temperature were four degrees of “hotter” and four degrees of
“colder.” Sounds fairly primitive, right? In 1724, Gabriel
Fahrenheit suggested a bit more precise system. He suggested that
mercury be used as the liquid in a thermometer, and calibrated his
scale using two reference points: the freezing point of water, and
the temperature of a healthy human being. He called his scale the
Fahrenheit scale, and measurements were in °F. In
1745, Carolus Linnaeus of Uppsala, Sweden, suggested that things
would be simpler if we made the scale range from 0 (at the freezing
point of water) to 100 (water's boiling point), and called this scale the
centigrade scale. (This scale was later abandoned in favor
of the Celsius scale, which is technically different from
centigrade in subtle ways that are not important here.)
Notice that all of these scales are relative—they are based
on the freezing point of water, which is an arbitrary (but
highly practical) reference point. A temperature reading of
$x{}^{\mathrm{o}}$
C basically means “
$x$
degrees hotter than the temperature
at which water freezes.” It wasn't until 1848, with the invention of
the Kelvin scale by Lord Kelvin, that mankind finally had an absolute
temperature scale. 0 K is the coldest possible temperature,
corresponding to
$273{}^{\mathrm{o}}$
C.
Loudness. Loudness is usually measured in
decibels (abbreviated dB). To be more precise,
decibels are used to measure the ratio of two
power levels. If we have two power levels
${P}_{1}$
and
${P}_{2}$
,
then the difference in decibels between the two
power levels is
$$10{\mathrm{log}}_{10}({P}_{2}/{P}_{1})\text{}\mathrm{d}\mathrm{B}.$$
So, if
${P}_{2}$
is about twice the level of
${P}_{1}$
, then the
difference is about 3 dB. Notice that this is a relative
system, providing a precise way to measure the
relative strength of two power levels, but not a
way to assign a number to one power level. In other words,
we haven't established any sort of absolute reference point.
(It's also a logarithmic scale, but that isn't important here.)
You may have used a mixer board, volume control knob, or digital
audio program that measures volume in dB. Normally, there's an
arbitrary point marked 0 dB, and then most of the readings have
negative values. In other words, 0 dB is the loudest volume,
and all other volume settings are softer.
None of these values are absolute—but how could they be? How
could your digital audio program know the absolute
loudness you will experience, which depends not only on the audio
data, but also the volume setting on your computer, the volume
knob on your amplifier, the power supplied by the amplifier
to your speakers, the distance you are from the speakers, and so on.
Sometimes people describe how loud something is in terms
of an absolute dB number. Following in the footsteps of
Gabriel Fahrenheit, this scale uses a reference point
based on the human body. “Absolute” dB numbers
are actually relative to the threshold of hearing for a normal
human.
Because of this, it's actually possible to have an
“absolute” dB reading that is negative. This simply means that
the intensity is below the threshold where most people are able
to hear it.
At this point, we should probably mention that there
is a way to devise an absolute scale for loudness,
by measuring a physical quantity such as pressure, energy,
or power, all of which have an absolute minimum value of zero.
The point is that these absolute systems aren't used in many
cases—the relative system is the one that's the most useful.
Velocity. How fast are you moving right now? Perhaps
you're sitting in a comfy chair, so you'd say that your
speed was zero. Maybe you're in a car and so you might say something
like 65 mph. (Hopefully someone else is driving!) Actually,
you are hurtling through space at almost 30 km per second!
That's about the speed that Earth travels in order to make the
939millionkm trek around the sun each year. Of course,
even this velocity is relative to the sun. Our solar system is
moving around within the Milky Way galaxy. So then how fast are
we actually moving, in absolute terms? Galileo told us back
in the 17th century that this question doesn't have an
answer—all velocity is relative.
Our difficulty in establishing absolute velocity is similar to
the difficulty in establishing position. After all,
velocity is displacement (difference between positions) over
time. To establish an absolute velocity, we'd need to have
some reference location that would “stay still” so that we
could measure our displacement from that location.
Unfortunately, everything in our universe seems to be
orbiting something else.
2.5Negating a Vector
The previous sections have presented a highlevel overview of vectors. The remainder of this
chapter looks at specific mathematical operations we perform on vectors. For each operation, we
first define the mathematical rules for performing the operation and then describe the geometric
interpretations of the operation and give some practical uses for the operation.
The first operation we'd like to consider is that of vector negation. When discussing the zero
vector, we asked you to recall from group theory the idea of the additive identity. Please
go back to wherever it was in your brain that you found the additive identity, perhaps between
the metaphorical couch cushions, or at the bottom of a box full of decadeold tax forms. Nearby,
you will probably find a similarly discarded obvioustothepointofuseless concept: the
additive inverse. Let's dust it off. For any group, the additive inverse of
$x$
, denoted
by
$x$
, is the element that yields the additive identity (zero) when added to
$x$
. Put simply,
$x+(x)=0$
. Another way of saying this is that elements in the group can be negated.
The negation operation can be applied to vectors. Every vector
$\mathbf{v}$
has an additive
inverse
$\mathbf{v}$
of the same dimension as
$\mathbf{v}$
such that
$\mathbf{v}+(\mathbf{v})=\mathbf{0}$
. (We will learn how to add vectors in Section 2.7.)
2.5.1Official Linear Algebra Rules
To negate a vector of any dimension, we simply negate each component of the vector. Stated
formally,
Negating a vector
$$\left[\begin{array}{c}{a}_{1}\\ {a}_{2}\\ \vdots \\ {a}_{n1}\\ {a}_{n}\end{array}\right]=\left[\begin{array}{c}{a}_{1}\\ {a}_{2}\\ \vdots \\ {a}_{n1}\\ {a}_{n}\end{array}\right].$$
Applying this to the specific cases of 2D, 3D, and 4D vectors, we have
Negating 2D, 3D, and 4D vectors
$$\begin{array}{rl}\left[\begin{array}{cc}x& y\end{array}\right]& =\left[\begin{array}{cc}x& y\end{array}\right],\\ \left[\begin{array}{ccc}x& y& z\end{array}\right]& =\left[\begin{array}{ccc}x& y& z\end{array}\right],\\ \left[\begin{array}{cccc}x& y& z& w\end{array}\right]& =\left[\begin{array}{cccc}x& y& z& w\end{array}\right].\end{array}$$
A few examples are
$$\begin{array}{rl}\left[\begin{array}{cc}4& 5\end{array}\right]& =\left[\begin{array}{cc}4& 5\end{array}\right],\\ \left[\begin{array}{ccc}1& 0& \sqrt{3}\end{array}\right]& =\left[\begin{array}{ccc}1& 0& \sqrt{3}\end{array}\right],\\ \left[\begin{array}{cccc}1.34& 3/4& 5& \pi \end{array}\right]& =\left[\begin{array}{cccc}1.34& 3/4& 5& \pi \end{array}\right].\end{array}$$
2.5.2Geometric Interpretation
Negating a vector results in a vector of the same magnitude but opposite direction, as shown in
Figure 2.8.
Remember, the position of a vector on a diagram is irrelevant—only the magnitude and direction
are important.
2.6Vector Multiplication by a Scalar
Although we cannot add a vector and a scalar, we can multiply a vector by a scalar. The result
is a vector that is parallel to the original vector, with a different length and possibly
opposite direction.
2.6.1Official Linear Algebra Rules
Vectortimesscalar multiplication is straightforward; we simply multiply each component of the
vector by the scalar. Stated formally,
Multiplying a vector by a scalar
$$k\left[\begin{array}{c}{a}_{1}\\ {a}_{2}\\ \vdots \\ {a}_{n1}\\ {a}_{n}\end{array}\right]=\left[\begin{array}{c}{a}_{1}\\ {a}_{2}\\ \vdots \\ {a}_{n1}\\ {a}_{n}\end{array}\right]k=\left[\begin{array}{c}k{a}_{1}\\ k{a}_{2}\\ \vdots \\ k{a}_{n1}\\ k{a}_{n}\end{array}\right].$$
Applying this rule to 3D vectors, as an example, we get
Multiplying a 3D vector by a scalar
$$k\left[\begin{array}{c}x\\ y\\ z\end{array}\right]=\left[\begin{array}{c}x\\ y\\ z\end{array}\right]k=\left[\begin{array}{c}kx\\ ky\\ kz\end{array}\right].$$
Although the scalar and vector may be written in either order, most people choose to put the scalar
on the left, preferring
$k\mathbf{v}$
to
$\mathbf{v}k$
.
A vector may also be divided by a nonzero scalar. This is equivalent to multiplying by the
reciprocal of the scalar:
Dividing a 3D vector by a scalar
$$\frac{\mathbf{v}}{k}=\left(\frac{1}{k}\right)\mathbf{v}=\left[\begin{array}{c}{v}_{x}/k\\ {v}_{y}/k\\ {v}_{z}/k\end{array}\right]\phantom{\rule{.25in}{0ex}}\text{for 3D vector}\mathbf{v}\text{and nonzero scalar}k.$$
Some examples are
$$\begin{array}{rl}2\left[\begin{array}{ccc}1& 2& 3\end{array}\right]& =\left[\begin{array}{ccc}2& 4& 6\end{array}\right],\\ 3\left[\begin{array}{cc}5.4& 0\end{array}\right]& =\left[\begin{array}{cc}16.2& 0\end{array}\right],\\ \left[\begin{array}{ccc}4.7& 6& 8\end{array}\right]/2& =\left[\begin{array}{ccc}2.35& 3& 4\end{array}\right].\end{array}$$
Here are a few things to notice about multiplication of a vector by a scalar:

When we multiply a vector and a scalar, we do not use any
multiplication symbol. The multiplication is signified by placing the
two quantities sidebyside (usually with the vector on the right).

Scalartimesvector multiplication and division both occur before any
addition and subtraction. For example
$3\mathbf{a}+\mathbf{b}$
is the same
as
$(3\mathbf{a})+\mathbf{b}$
, not
$3(\mathbf{a}+\mathbf{b})$
.

A scalar may not be divided by a vector, and a vector may not be
divided by another vector.

Vector negation can be viewed as the special case of multiplying a
vector by the scalar
$1$
.
2.6.2Geometric Interpretation
Geometrically, multiplying a vector by a scalar
$k$
has the effect of scaling the length by a
factor of
$\leftk\right$
. For example, to double the length of a vector we would multiply the vector
by 2. If
$k<0$
, then the direction of the vector is flipped.
Figure 2.9 illustrates a vector multiplied by several
different scalars.
2.7Vector Addition and Subtraction
We can add and subtract two vectors, provided they are of the same dimension. The result is a
vector quantity of the same dimension as the vector operands. We use the same notation for
vector addition and subtraction as is used for addition and subtraction of scalars.
2.7.1Official Linear Algebra Rules
The linear algebra rules for vector addition are simple: to add two vectors, we add the
corresponding components:
Adding two vectors
$$\left[\begin{array}{c}{a}_{1}\\ {a}_{2}\\ \vdots \\ {a}_{n1}\\ {a}_{n}\end{array}\right]+\left[\begin{array}{c}{b}_{1}\\ {b}_{2}\\ \vdots \\ {b}_{n1}\\ {b}_{n}\end{array}\right]=\left[\begin{array}{c}{a}_{1}+{b}_{1}\\ {a}_{2}+{b}_{2}\\ \vdots \\ {a}_{n1}+{b}_{n1}\\ {a}_{n}+{b}_{n}\end{array}\right].$$
Subtraction can be interpreted as adding the negative, so
$\mathbf{a}\mathbf{b}=\mathbf{a}+(\mathbf{b})$
:
Subtracting two vectors
$$\left[\begin{array}{c}{a}_{1}\\ {a}_{2}\\ \vdots \\ {a}_{n1}\\ {a}_{n}\end{array}\right]\left[\begin{array}{c}{b}_{1}\\ {b}_{2}\\ \vdots \\ {b}_{n1}\\ {b}_{n}\end{array}\right]=\left[\begin{array}{c}{a}_{1}\\ {a}_{2}\\ \vdots \\ {a}_{n1}\\ {a}_{n}\end{array}\right]+(\left[\begin{array}{c}{b}_{1}\\ {b}_{2}\\ \vdots \\ {b}_{n1}\\ {b}_{n}\end{array}\right])=\left[\begin{array}{c}{a}_{1}{b}_{1}\\ {a}_{2}{b}_{2}\\ \vdots \\ {a}_{n1}{b}_{n1}\\ {a}_{n}{b}_{n}\end{array}\right].$$
For example, given
$$\mathbf{a}=\left[\begin{array}{c}1\\ 2\\ 3\end{array}\right],\phantom{\rule{.25in}{0ex}}\mathbf{b}=\left[\begin{array}{c}4\\ 5\\ 6\end{array}\right],\phantom{\rule{.25in}{0ex}}\mathbf{c}=\left[\begin{array}{c}7\\ 3\\ 0\end{array}\right],$$
then
$$\begin{array}{rl}\mathbf{a}+\mathbf{b}& =\left[\begin{array}{c}1\\ 2\\ 3\end{array}\right]+\left[\begin{array}{c}4\\ 5\\ 6\end{array}\right]=\left[\begin{array}{c}1+4\\ 2+5\\ 3+6\end{array}\right]=\left[\begin{array}{c}5\\ 7\\ 9\end{array}\right],\\ \mathbf{a}\mathbf{b}& =\left[\begin{array}{c}1\\ 2\\ 3\end{array}\right]\left[\begin{array}{c}4\\ 5\\ 6\end{array}\right]=\left[\begin{array}{c}14\\ 25\\ 36\end{array}\right]=\left[\begin{array}{c}3\\ 3\\ 3\end{array}\right],\\ \mathbf{b}+\mathbf{c}\mathbf{a}& =\left[\begin{array}{c}4\\ 5\\ 6\end{array}\right]+\left[\begin{array}{c}7\\ 3\\ 0\end{array}\right]\left[\begin{array}{c}1\\ 2\\ 3\end{array}\right]=\left[\begin{array}{c}4+71\\ 5+(3)2\\ 6+03\end{array}\right]=\left[\begin{array}{c}10\\ 0\\ 3\end{array}\right].\end{array}$$
A vector cannot be added or subtracted with a scalar, or with a vector of a different dimension.
Also, just like addition and subtraction of scalars, vector addition is commutative,
$$\mathbf{a}+\mathbf{b}=\mathbf{b}+\mathbf{a},$$
whereas vector subtraction is anticommutative,
$$\mathbf{a}\mathbf{b}=(\mathbf{b}\mathbf{a}).$$
2.7.2Geometric Interpretation
We can add vectors
$\mathbf{a}$
and
$\mathbf{b}$
geometrically by positioning the vectors so that
the head of
$\mathbf{a}$
touches the tail of
$\mathbf{b}$
and then drawing a vector from the tail
of
$\mathbf{a}$
to the head of
$\mathbf{b}$
. In other words, if we start at a point and apply
the displacements specified by
$\mathbf{a}$
and then
$\mathbf{b}$
, it's the same as if we had
applied the single displacement
$\mathbf{a}+\mathbf{b}$
. This is known as the
triangle rule of vector addition. It also works for vector subtraction, as shown in
Figure 2.10.
Figure 2.10 provides geometric evidence that vector addition is commutative but
vector subtraction is not. Notice that the vector labeled
$\mathbf{a}+\mathbf{b}$
is identical to
the vector labeled
$\mathbf{b}+\mathbf{a}$
, but the vectors
$\mathbf{d}\mathbf{c}$
and
$\mathbf{c}\mathbf{d}$
point in opposite directions because
$\mathbf{d}\mathbf{c}=(\mathbf{c}\mathbf{d})$
.
The triangle rule can be extended to more than two vectors. Figure 2.11 shows how
the triangle rule verifies something we stated in Section 2.3.1: a
vector can be interpreted as a sequence of axially aligned
displacements.
Figure 2.12 is a reproduction of
Figure 2.5,
which shows how the vector
$[1,3,4]$
may be
interpreted as a displacement of 1 unit to the right, 3 units down, and then 4 units forward, and
can be verified mathematically by using vector addition:
$$\left[\begin{array}{c}1\\ 3\\ 4\end{array}\right]=\left[\begin{array}{c}1\\ 0\\ 0\end{array}\right]+\left[\begin{array}{c}0\\ 3\\ 0\end{array}\right]+\left[\begin{array}{c}0\\ 0\\ 4\end{array}\right].$$
This seems obvious, but this is a very powerful concept. We will use a similar technique in
Section 4.2 to transform vectors from one coordinate space to
another.
2.7.3Displacement Vector from One Point to Another
It is very common that we will need to compute the displacement from one point to another. In
this case, we can use the triangle rule and vector subtraction.
Figure 2.13 shows how the displacement vector from
$\mathbf{a}$
to
$\mathbf{b}$
can be computed by subtracting
$\mathbf{a}$
from
$\mathbf{b}$
.
As Figure 2.13 shows, to compute the vector from
$\mathbf{a}$
to
$\mathbf{b}$
, we interpret the points
$\mathbf{a}$
and
$\mathbf{b}$
as vectors from the origin,
and then use the triangle rule. In fact, this is how vectors are defined in some texts: the
subtraction of two points.
Notice that the vector subtraction
$\mathbf{b}\mathbf{a}$
yields a vector from
$\mathbf{a}$
to
$\mathbf{b}$
. It doesn't make any sense to simply find the vector “between two
points,” since the language in this sentence does not specify a direction. We must always form
a vector that goes from one point to another point.
2.8Vector Magnitude (Length)
As we have discussed, vectors have magnitude and direction. However, you might have noticed that
neither the magnitude nor the direction is expressed explicitly in the vector (at least not when
we use Cartesian coordinates). For example, the magnitude of the 2D vector
$[3,4]$
is neither 3
nor 4; it's 5. Since the magnitude of the vector is not expressed explicitly, we must compute it.
The magnitude of a vector is also known as the length or norm of the vector.
2.8.1Official Linear Algebra Rules
In linear algebra, the magnitude of a vector is denoted by using double vertical bars surrounding
the vector. This is similar to the single vertical bar notation used for the absolute value
operation for scalars. This notation and the equation for computing the magnitude of a vector of
arbitrary dimension
$n$
are shown in Equation (2.2):
Magnitude of a vector of arbitrary dimension
$$\begin{array}{}\text{(2.2)}& \parallel \mathbf{v}\parallel =\sqrt{\sum _{i=1}^{n}{{v}_{i}}^{2}}=\sqrt{{{v}_{1}}^{2}+{{v}_{2}}^{2}+\cdots +{{v}_{n1}}^{2}+{{v}_{n}}^{2}}.\end{array}$$
Thus, the magnitude of a vector is the square root of the sum of the squares of the components of
the vector. This sounds complicated, but the magnitude equations for 2D and 3D vectors are
actually very simple:
Vector magnitude for 2D and 3D vectors
$$\begin{array}{rlr}\text{(2.3)}& \parallel \mathbf{v}\parallel & =\sqrt{{{v}_{x}}^{2}+{{v}_{y}}^{2}}& (\text{for a 2D vector}\mathbf{v}),\parallel \mathbf{v}\parallel & =\sqrt{{{v}_{x}}^{2}+{{v}_{y}}^{2}+{{v}_{z}}^{2}}& (\text{for a 3D vector}\mathbf{v}).\end{array}$$
The magnitude of a vector is a nonnegative scalar quantity. An example of how to compute the
magnitude of a 3D vector is
$$\begin{array}{r}\begin{array}{rl}\parallel \left[\begin{array}{ccc}5& 4& 7\end{array}\right]\parallel & =\sqrt{{5}^{2}+{(4)}^{2}+{7}^{2}}=\sqrt{25+16+49}=\sqrt{90}\\ & \approx 9.4868.\end{array}\end{array}$$
Some books use a single bar notation to indicate vector
magnitude:
$\left\mathbf{v}\right$
One quick note to satisfy all you sticklers who already know about vector norms and at this
moment are pointing your web browser to gamemath.com, looking for the email address for errata. The
term norm actually has a very general definition, and basically any equation that meets a
certain set of criteria can call itself a norm. So to describe Equation (2.2)
as the equation for the vector norm is slightly misleading. To be more accurate, we
should say that Equation (2.2) is the equation for the 2norm, which is
one specific way to calculate a norm. The 2norm belongs to a class of norms known as the
$p$
norms, and the
$p$
norm is not the only way to define a norm. Still, omitting this
level of generality isn't too harmful of a delusion; because the 2norm measures Euclidian
distance, it is by far the most commonly used norm in geometric applications. It is also widely
used in situations even where a geometric interpretation is not directly applicable. Readers
interested in such exotica should check out Exercise 15.
2.8.2Geometric Interpretation
Let's try to get a better understanding of why Equation (2.3) works. For any
vector
$\mathbf{v}$
in 2D, we can form a right triangle with
$\mathbf{v}$
as the hypotenuse, as
shown in Figure 2.14.
Notice that to be precise we had to put absolute value signs around the components
${v}_{x}$
and
${v}_{y}$
. The components of the vector may be negative, since they are signed
displacements, but length is always positive.
The Pythagorean theorem states that for any right triangle, the square of the length of the
hypotenuse is equal to the sum of the squares of the lengths of the other two sides. Applying
this theorem to Figure 2.14, we have
$${\parallel \mathbf{v}\parallel}^{2}={\left{v}_{x}\right}^{2}+{\left{v}_{y}\right}^{2}.$$
Since
${\leftx\right}^{2}={x}^{2}$
, we can omit the absolute value symbols:
$${\parallel \mathbf{v}\parallel}^{2}={{v}_{x}}^{2}+{{v}_{y}}^{2}.$$
Then, by taking the square root of both sides and simplifying, we get
$$\begin{array}{rl}\sqrt{{\parallel \mathbf{v}\parallel}^{2}}& =\sqrt{{{v}_{x}}^{2}+{{v}_{y}}^{2}},\\ \parallel \mathbf{v}\parallel & =\sqrt{{{v}_{x}}^{2}+{{v}_{y}}^{2}},\end{array}$$
which is the same as Equation (2.3). The proof of the magnitude equation in 3D
is only slightly more complicated.
For any positive magnitude
$m$
, there are an infinite number of vectors of magnitude
$m$
. Since
these vectors all have the same length but different directions, they form a circle when the tails
are placed at the origin, as shown in Figure 2.15.
2.9Unit Vectors
For many vector quantities, we are concerned only with direction and not magnitude: “Which way
am I facing?” “Which way is the surface oriented?” In these cases, it is often convenient to
use unit vectors. A unit vector is a vector that has a magnitude of one. Unit vectors are
also known as normalized vectors.
Unit vectors are also sometimes simply called normals; however, a warning is in order
concerning terminology. The word “normal” carries with it the connotation of “perpendicular.”
When most people speak of a “normal” vector, they are usually referring to a vector that is
perpendicular to something. For example, a surface normal at a given point on an object is a
vector that is perpendicular to the surface at that location. However, since the concept of
perpendicular is related only to the direction of a vector and not its magnitude, in most cases you
will find that unit vectors are used for normals instead of a vector of arbitrary length. When
this book refers to a vector as a “normal,” it means “a unit vector perpendicular to something
else.” This is common usage, but be warned that the word “normal” primarily means
“perpendicular” and not “unit length.” Since it is so common for normals to be unit vectors,
we will take care to call out any situation where a “normal” vector does not have unit length.
In summary, a “normalized” vector always has unit length, but a “normal” vector is a vector
that is perpendicular to something and by convention usually has unit length.
2.9.1Official Linear Algebra Rules
For any nonzero vector
$\mathbf{v}$
, we can compute a unit vector that points in the same direction
as
$\mathbf{v}$
. This process is known as normalizing the vector. In this book we use a
common notation of putting a hat symbol over unit vectors; for example,
$\hat{\mathbf{v}}$
(pronounced
“v hat”). To normalize a vector, we divide the vector by its magnitude:
Normalizing a vector
$$\hat{\mathbf{v}}=\frac{\mathbf{v}}{\parallel \mathbf{v}\parallel}\phantom{\rule{.5in}{0ex}}\text{for any nonzero vector}\mathbf{v}.$$
For example, to normalize the 2D vector
$[12,5]$
,
$$\begin{array}{r}\begin{array}{rl}\frac{\left[\begin{array}{cc}12& 5\end{array}\right]}{\parallel \left[\begin{array}{cc}12& 5\end{array}\right]\parallel}& =\frac{\left[\begin{array}{cc}12& 5\end{array}\right]}{\sqrt{{12}^{2}+{5}^{2}}}=\frac{\left[\begin{array}{cc}12& 5\end{array}\right]}{\sqrt{169}}=\frac{\left[\begin{array}{cc}12& 5\end{array}\right]}{13}=\left[\begin{array}{cc}{\displaystyle \frac{12}{13}}& {\displaystyle \frac{5}{13}}\end{array}\right]\\ & \approx \left[\begin{array}{cc}0.923& 0.385\end{array}\right].\end{array}\end{array}$$
The zero vector cannot be normalized. Mathematically, this is not allowed because it would result
in division by zero. Geometrically, it makes sense because the zero vector does not define a
direction—if we normalized the zero vector, in what direction should the resulting vector point?
2.9.2Geometric Interpretation
In 2D, if we draw a unit vector with the tail at the origin, the head of the vector will touch a
unit circle centered at the origin. (A unit circle has a radius of 1.) In 3D, unit vectors touch
the surface of a unit sphere. Figure 2.16 shows several 2D vectors of arbitrary
length in gray, beneath their normalized counterparts in black.
Notice that normalizing a vector makes some vectors shorter (if their length was greater than 1)
and some vectors longer (if their length was less than 1).
We are now prepared to derive one of the oldest and most
fundamental formulas in computational geometry: the distance formula. This formula is used to
compute the distance between two points.
First, let's define distance as the length of the line segment between the two points. Since a
vector is a directed line segment, geometrically it makes sense that the distance between the two
points would be equal to the length of a vector from one point to the other. Let's derive the
distance formula in 3D. First, we will compute the vector
$\mathbf{d}$
from
$\mathbf{a}$
to
$\mathbf{b}$
. We learned how to do this in 2D in Section 2.7.3. In 3D, we use
$$\mathbf{d}=\mathbf{b}\mathbf{a}=\left[\begin{array}{c}{b}_{x}{a}_{x}\\ {b}_{y}{a}_{y}\\ {b}_{z}{a}_{z}\end{array}\right].$$
The distance between
$\mathbf{a}$
and
$\mathbf{b}$
is equal to the length of the vector
$\mathbf{d}$
, which we computed in Section 2.8:
$$\mathrm{d}\mathrm{i}\mathrm{s}\mathrm{t}\mathrm{a}\mathrm{n}\mathrm{c}\mathrm{e}(\mathbf{a},\mathbf{b})=\parallel \mathbf{d}\parallel =\sqrt{{{d}_{x}}^{2}+{{d}_{y}}^{2}+{{d}_{z}}^{2}}.$$
Substituting for
$\mathbf{d}$
, we get
The 3D distance formula
$$\mathrm{d}\mathrm{i}\mathrm{s}\mathrm{t}\mathrm{a}\mathrm{n}\mathrm{c}\mathrm{e}(\mathbf{a},\mathbf{b})=\parallel \mathbf{b}\mathbf{a}\parallel =\sqrt{({b}_{x}{a}_{x}{)}^{2}+({b}_{y}{a}_{y}{)}^{2}+({b}_{z}{a}_{z}{)}^{2}}.$$
Thus, we have derived the distance formula in 3D. The 2D equation is even simpler:
The 2D distance formula
$$\mathrm{d}\mathrm{i}\mathrm{s}\mathrm{t}\mathrm{a}\mathrm{n}\mathrm{c}\mathrm{e}(\mathbf{a},\mathbf{b})=\parallel \mathbf{b}\mathbf{a}\parallel =\sqrt{({b}_{x}{a}_{x}{)}^{2}+({b}_{y}{a}_{y}{)}^{2}}.$$
Let's look at an example in 2D:
$$\begin{array}{r}\begin{array}{rl}\mathrm{d}\mathrm{i}\mathrm{s}\mathrm{t}\mathrm{a}\mathrm{n}\mathrm{c}\mathrm{e}(\left[\begin{array}{cc}5& 0\end{array}\right],\left[\begin{array}{cc}1& 8\end{array}\right])& =\sqrt{(15{)}^{2}+(80{)}^{2}}\\ & =\sqrt{(6{)}^{2}+{8}^{2}}=\sqrt{100}=10.\end{array}\end{array}$$
Notice that it doesn't matter which point we call
$\mathbf{a}$
and which point we call
$\mathbf{b}$
. If we define
$\mathbf{d}$
to be the vector from
$\mathbf{b}$
to
$\mathbf{a}$
instead of from
$\mathbf{a}$
to
$\mathbf{b}$
, we will derive a slightly different, but
mathematically equivalent, equation.
2.11Vector Dot Product
Section 2.6 showed how to multiply a vector by a scalar. We can also multiply
two vectors together. There are two types of vector products. The first vector product is the
dot product (also known as the inner product), the subject of this section. We talk
about the other vector product, the cross product, in Section 2.12.
The dot product is ubiquitous in video game programming, useful in everything from graphics, to
simulation, to AI. Following the pattern we used for the operations, we first discuss the
algebraic rules for computing dot products in Section 2.11.1, followed by some
geometric interpretations in Section 2.11.2.
The dot product formula is one of the few formulas in this book worth memorizing. First of all,
it's really easy to memorize. Also, if you understand what the dot product does, the formula
makes sense. Furthermore, the dot product has important relationships to many other operations,
such as matrix multiplication, convolution of signals, statistical correlations, and Fourier
transforms. Understanding the formula will make these relationships more apparent.
Even more important than memorizing a formula is to get an intuitive grasp for what the dot product
does. If there is only enough space in your brain for either the formula or the geometric
definition, then we recommend internalizing the geometry, and getting the formula tattooed on your
hand. You need to understand the geometric definition in order to use the dot product. When
programming in computer languages such as C++, HLSL, or even Matlab and Maple, you won't need to
know the formula anyway, since you will usually tell the computer to do a dot product calculation
not by typing in the formula, but by invoking a highlevel function or overloaded operator.
Furthermore, the geometric definition of the dot product does not assume any particular coordinate
frame or even the use of Cartesian coordinates.
2.11.1Official Linear Algebra Rules
The name “dot product” comes from the dot symbol used in the notation:
$\mathbf{a}\cdot \mathbf{b}$
. Just like scalartimesvector multiplication, the vector dot product is performed
before addition and subtraction, unless parentheses are used to override this default order of
operations. Note that although we usually omit the multiplication symbol when multiplying two
scalars or a scalar and a vector, we must not omit the dot symbol when performing a vector dot
product.
If you ever see two vectors placed sidebyside with no symbol in between, interpret this
according to the rules of matrix multiplication, which we discuss in
Chapter 4.
The dot product of two vectors is the sum of the products of corresponding components, resulting in
a scalar:
Vector dot product
$$\left[\begin{array}{c}{a}_{1}\\ {a}_{2}\\ \vdots \\ {a}_{n1}\\ {a}_{n}\end{array}\right]\cdot \left[\begin{array}{c}{b}_{1}\\ {b}_{2}\\ \vdots \\ {b}_{n1}\\ {b}_{n}\end{array}\right]={a}_{1}{b}_{1}+{a}_{2}{b}_{2}+\cdots +{a}_{n1}{b}_{n1}+{a}_{n}{b}_{n}.$$
This can be expressed succinctly by using the summation notation
Dot product using summation notation
$$\mathbf{a}\cdot \mathbf{b}=\sum _{i=1}^{n}{a}_{i}{b}_{i}.$$
Applying these rules to the 2D and 3D cases yields
2D and 3D dot products
$$\begin{array}{rlrl}\mathbf{a}\cdot \mathbf{b}& ={a}_{x}{b}_{x}+{a}_{y}{b}_{y}& & (\mathbf{a}\text{and}\mathbf{b}\text{are 2D vectors}),\\ \mathbf{a}\cdot \mathbf{b}& ={a}_{x}{b}_{x}+{a}_{y}{b}_{y}+{a}_{z}{b}_{z}& & (\mathbf{a}\text{and}\mathbf{b}\text{are 3D vectors}).\end{array}$$
Examples of the dot product in 2D and 3D are
$$\begin{array}{rl}\left[\begin{array}{cc}4& 6\end{array}\right]\cdot \left[\begin{array}{cc}3& 7\end{array}\right]& =(4)(3)+(6)(7)=30,\\ \left[\begin{array}{c}3\\ 2\\ 7\end{array}\right]\cdot \left[\begin{array}{c}0\\ 4\\ 1\end{array}\right]& =(3)(0)+(2)(4)+(7)(1)=15.\end{array}$$
It is obvious from inspection of the equations that vector dot product is commutative:
$\mathbf{a}\cdot \mathbf{b}=\mathbf{b}\cdot \mathbf{a}$
. More vector algebra laws concerning the dot
product are given in Section 2.13.
2.11.2Geometric Interpretation
Now let's discuss the more important aspect of the dot product: what it means geometrically. It
would be difficult to make too big of a deal out of the dot product, as it is fundamental to almost
every aspect of 3D math. Because of its supreme importance, we're going to dwell on it a bit.
We'll discuss two slightly different ways of thinking about this operation geometrically; since
they are really equivalent, you may or may not think one interpretation or the other is “more
fundamental,” or perhaps you may think we are being redundant and wasting your time. You might
especially think this if you already have some exposure to the dot product, but please indulge us.
The first geometric definition to present is perhaps the less common of the two, but in agreement
with the advice of Dray and Manogue [1], we believe it's actually the more
useful. The interpretation we first consider is that of the dot product performing a
projection.
Assume for the moment that
$\hat{\mathbf{a}}$
is a unit vector, and
$\mathbf{b}$
is a vector of any
length. Now take
$\mathbf{b}$
and project it onto a line parallel to
$\hat{\mathbf{a}}$
, as in
Figure 2.17.
(Remember that vectors are displacements and do not have a fixed
position, so we are free to move them around on a diagram anywhere we wish.) We can define the dot
product
$\hat{\mathbf{a}}\cdot \mathbf{b}$
as the signed length of the projection of
$\mathbf{b}$
onto
this line. The term “projection” has a few different technical meanings (see
Section 5.3) and we won't bother attempting a formal
definition here. You can think of the projection of
$\mathbf{b}$
onto
$\hat{\mathbf{a}}$
as the “shadow” that
$\mathbf{b}$
casts on
$\hat{\mathbf{a}}$
when the
rays of light are perpendicular to
$\hat{\mathbf{a}}$
.
We have drawn the projections as arrows, but remember that the result of a dot product is a scalar,
not a vector. Still, when you first learned about negative numbers, your teacher probably depicted
numbers as arrows on a number line, to emphasize their sign, just as we have. After all, a scalar
is a perfectly valid onedimensional vector.
What does it mean for the dot product to measure a signed length? It means the value will
be negative when the projection of
$\mathbf{b}$
points in the opposite direction from
$\hat{\mathbf{a}}$
, and the projection has zero length (it is a single point) when
$\hat{\mathbf{a}}$
and
$\mathbf{b}$
are perpendicular. These cases are illustrated in
Figure 2.18.
In other words, the sign of the dot product can give us a rough classification of the relative
directions of the two vectors. Imagine a line (in 2D) or plane (in 3D) perpendicular to the
vector
$\hat{\mathbf{a}}$
. The sign of the dot product
$\hat{\mathbf{a}}\cdot \mathbf{b}$
tells us which
halfspace
$\mathbf{b}$
lies in. This is illustrated in Figure 2.19.
Next, consider what happens when we scale
$\mathbf{b}$
by some factor
$k$
. As shown in
Figure 2.20, the length of the projection (and thus the value of the dot
product) increases by the same factor. The two triangles have equal interior angles and thus are
similar. Since the hypotenuse on the right is longer than the hypotenuse on the left by a factor of
$k$
, by the properties of similar triangles, the base on the right is also longer by a factor of
$k$
.
Let's state this fact algebraically and prove it by using the formula:
Dot product is associative with multiplication by a scalar
$$\begin{array}{r}\begin{array}{rl}\hat{\mathbf{a}}\cdot (k\mathbf{b})& ={a}_{x}(k{b}_{x})+{a}_{y}(k{b}_{y})+{a}_{z}(k{b}_{z})\\ & =k({a}_{x}{b}_{x}+{a}_{y}{b}_{y}+{a}_{z}{b}_{z})\\ & =k(\hat{\mathbf{a}}\cdot \mathbf{b}).\end{array}\end{array}$$
The expanded scalar math in the middle uses three dimensions as our example, but the vector
notation at either end of the equation applies for vectors of any dimension.
We've seen what happens when we scale
$\mathbf{b}$
: the length of its projection onto
$\hat{\mathbf{a}}$
increases along with the value of the dot product. What if we scale
$\mathbf{a}$
? The algebraic
argument we just made can be used to show that the value of the dot product scales with the length
of
$\mathbf{a}$
, just like it does when we scale
$\mathbf{b}$
. In other words,
Dot product is associative with multiplication by a scalar for either vector
$$(k\mathbf{a})\cdot \mathbf{b}=k(\mathbf{a}\cdot \mathbf{b})=\mathbf{a}\cdot (k\mathbf{b}).$$
So scaling
$\mathbf{a}$
scales the numeric value of the dot product. However, this scale has no
affect geometrically on the length of the projection of
$\mathbf{b}$
onto
$\mathbf{a}$
. Now that
we know what happens if we scale either
$\mathbf{a}$
or
$\mathbf{b}$
, we can write our geometric
definition without any assumptions about the length of the vectors.
Dot Product as Projection
The dot product
$\mathbf{a}\cdot \mathbf{b}$
is equal to the signed length of the projection of
$\mathbf{b}$
onto any line parallel to
$\mathbf{a}$
, multiplied by the length of
$\mathbf{a}$
.
As we continue to examine the properties of the dot product, some will be easiest to illustrate
geometrically when either
$\mathbf{a}$
, or both
$\mathbf{a}$
and
$\mathbf{b}$
, are unit vectors.
Because we have shown that scaling either
$\mathbf{a}$
or
$\mathbf{b}$
directly scales the value of
the dot product, it will be easy to generalize our results after we have obtained them.
Furthermore, in the algebraic arguments that accompany each geometric argument, unit vector
assumptions won't be necessary. Remember that we put hats on top of vectors that are assumed to
have unit length.
You may well wonder why the dot product measures the projection of the second operand onto the
first, and not the other way around. When the two vectors
$\hat{\mathbf{a}}$
and
$\hat{\mathbf{b}}$
are unit
vectors, we can easily make a geometric argument that the projection of
$\hat{\mathbf{a}}$
onto
$\hat{\mathbf{b}}$
has the same length as the projection of
$\hat{\mathbf{b}}$
onto
$\hat{\mathbf{a}}$
. Consider
Figure 2.21. The two triangles have equal interior angles and thus are
similar. Since
$\hat{\mathbf{a}}$
and
$\hat{\mathbf{b}}$
are corresponding sides and have the same length,
the two triangles are reflections of each other.
We've already shown how scaling either vector will scale the dot product proportionally, so this
result applies for
$\mathbf{a}$
and
$\mathbf{b}$
with arbitrary length. Furthermore, this
geometric fact is also trivially verified by using the formula, which does not depend on the
assumption that the vectors have equal length. Using two dimensions as our example this time,
$$\mathbf{a}\cdot \mathbf{b}={a}_{x}{b}_{x}+{a}_{y}{b}_{y}={b}_{x}{a}_{x}+{b}_{y}{a}_{y}=\mathbf{b}\cdot \mathbf{a}.$$
Dot product is commutativeThe next important property of the dot product is that it distributes over addition and
subtraction, just like scalar multiplication. This time let's do the algebra before the geometry.
When we say that the dot product “distributes,” that means that if one of the operands to the
dot product is a sum, then we can take the dot product of the pieces individually, and then take
their sum. Switching back to three dimensions for our example,
Dot product distributes over addition and subtraction
$$\begin{array}{rl}\mathbf{a}\cdot (\mathbf{b}+\mathbf{c})& =\left[\begin{array}{c}{a}_{x}\\ {a}_{y}\\ {a}_{z}\end{array}\right]\cdot \left[\begin{array}{c}{b}_{x}+{c}_{x}\\ {b}_{y}+{c}_{y}\\ {b}_{z}+{c}_{z}\end{array}\right]\\ & ={a}_{x}({b}_{x}+{c}_{x})+{a}_{y}({b}_{y}+{c}_{y})+{a}_{z}({b}_{z}+{c}_{z})\\ & ={a}_{x}{b}_{x}+{a}_{x}{c}_{x}+{a}_{y}{b}_{y}+{a}_{y}{c}_{y}+{a}_{z}{b}_{z}+{a}_{z}{c}_{z}\\ & =({a}_{x}{b}_{x}+{a}_{y}{b}_{y}+{a}_{z}{b}_{z})+({a}_{x}{c}_{x}+{a}_{y}{c}_{y}+{a}_{z}{c}_{z})\\ & =\mathbf{a}\cdot \mathbf{b}+\mathbf{a}\cdot \mathbf{c}.\end{array}$$
By replacing
$\mathbf{c}$
with
$\mathbf{c}$
, it's clear that the dot product distributes over
vector subtraction just as it does for vector addition. Figure 2.22
shows how the dot product distributes over addition.
Now let's look at a special situation in which one of the vectors is the unit vector pointing in
the
$+x$
direction, which we'll denote as
$\hat{\mathbf{x}}$
. As shown in
Figure 2.23, the signed length of the projection is simply the
$x$
coordinate of
the original vector. In other words, taking the dot product of a vector with a cardinal axis
“sifts” out the coordinate for that axis.
If we combine this “sifting” property of the dot product with the fact that it distributes over
addition, which we have been able to show in purely geometric terms, we can see why the formula
has to be what it is.
Because the dot product measures the length of a projection, it has an interesting relationship
to the vector magnitude calculation. Remember that the vector magnitude is a scalar measuring
the amount of displacement (the length) of the vector. The dot product also measures the amount
of displacement, but only the displacement in a particular direction is counted;
perpendicular displacement is discarded by the projecting process. But what if we measure the
displacement in the same direction that the vector is pointing? In this case, all of the
vector's displacement is in the direction being measured, so if we project a vector onto itself,
the length of that projection is simply the magnitude of the vector. But remember that
$\mathbf{a}\cdot \mathbf{b}$
is equal to the length of the projection of
$\mathbf{b}$
onto
$\mathbf{a}$
, scaled by
$\parallel \mathbf{a}\parallel $
. If we dot a vector with itself, such as
$\mathbf{v}\cdot \mathbf{v}$
, we get the length of the projection, which is
$\parallel \mathbf{v}\parallel $
, times the length of the vector we are projecting onto, which is also
$\parallel \mathbf{v}\parallel $
. In other words,
Relationship between vector magnitude and the dot product
$$\begin{array}{rlrl}\mathbf{v}\cdot \mathbf{v}& ={\parallel \mathbf{v}\parallel}^{2},& \parallel \mathbf{v}\parallel & =\sqrt{\mathbf{v}\cdot \mathbf{v}}.\end{array}$$
Before we switch to the second interpretation of the dot product, let's check out one more very
common use of the dot product as a projection. Assume once more that
$\hat{\mathbf{a}}$
is a unit
vector and
$\mathbf{b}$
has arbitrary length. Using the dot product, it's possible to separate
$\mathbf{b}$
into two values,
${\mathbf{b}}_{\parallel}$
and
${\mathbf{b}}_{\perp}$
(read “
$\mathbf{b}$
parallel” and
“
$\mathbf{b}$
perp”), which are parallel and perpendicular to
$\hat{\mathbf{a}}$
, respectively, such
that
$\mathbf{b}={\mathbf{b}}_{\parallel}+{\mathbf{b}}_{\perp}$
. Figure 2.24 illustrates the geometry
involved.
We've already established that the length of
${\mathbf{b}}_{\parallel}$
will be equal to
$\hat{\mathbf{a}}\cdot \mathbf{b}$
. But the dot product yields a scalar, and
${\mathbf{b}}_{\parallel}$
is a vector, so we'll take
the direction specified by the unit vector
$\hat{\mathbf{a}}$
and scale it up:
$${\mathbf{b}}_{\parallel}=(\hat{\mathbf{a}}\cdot \mathbf{b})\hat{\mathbf{a}}.$$
Once we know
${\mathbf{b}}_{\parallel}$
, we can easily solve for
${\mathbf{b}}_{\perp}$
:
$$\begin{array}{rl}{\mathbf{b}}_{\perp}+{\mathbf{b}}_{\parallel}& =\mathbf{b},\\ {\mathbf{b}}_{\perp}& =\mathbf{b}{\mathbf{b}}_{\parallel},\\ {\mathbf{b}}_{\perp}& =\mathbf{b}(\hat{\mathbf{a}}\cdot \mathbf{b})\hat{\mathbf{a}}.\end{array}$$
It's not too difficult to generalize these results to the case where
$\mathbf{a}$
is not a unit
vector.
In the rest of this book, we make use of these equations several times to separate a vector
into components that are parallel and perpendicular to another vector.
Now let's examine the dot product through the lens of trigonometry. This is the more common
geometric interpretation of the dot product, which places a bit more emphasis on the angle between
the vectors. We've been thinking in terms of projections, so we haven't had much need for this
angle. Less experienced and conscientious authors [2] might give you just one of
the two important viewpoints, which is probably sufficient to interpret an equation that contains
the dot product. However, a more valuable skill is to recognize situations for which the dot
product is the correct tool for the job; sometimes it helps to have other interpretations pointed
out, even if they are “obviously” equivalent to each other.
Consider the right triangle on the righthand side of Figure 2.25. As the
figure shows, the length of the hypotenuse is 1 (since
$\hat{\mathbf{b}}$
is a unit vector) and the
length of the base is equal to the dot product
$\hat{\mathbf{a}}\cdot \hat{\mathbf{b}}$
. From elementary
trig (which was reviewed in Section 1.4.4), remember that the cosine
of an angle is the ratio of the length of the adjacent leg divided by the length of the hypotenuse.
Plugging in the values from Figure 2.25, we have
$$\begin{array}{rl}\mathrm{cos}\theta & =\frac{\text{adjacent}}{\text{hypotenuse}}=\frac{\hat{\mathbf{a}}\cdot \hat{\mathbf{b}}}{1}=\hat{\mathbf{a}}\cdot \hat{\mathbf{b}}.\end{array}$$
In other words, the dot product of two unit vectors is equal to the cosine of the angle between
them. This statement is true even if the right triangle in Figure 2.25
cannot be formed, when
$\hat{\mathbf{a}}\cdot \hat{\mathbf{b}}\le 0$
and
$\theta >90{}^{\mathrm{o}}$
. Remember
that the dot product of any vector with the vector
$\hat{\mathbf{x}}=[1,0,0]$
will simply extract the
$x$
coordinate of the vector. In fact, the
$x$
coordinate of a unit vector that has been rotated by
an angle of
$\theta $
from standard position is one way to define the value of
$\mathrm{cos}\theta $
.
Review Section 1.4.4 if this isn't fresh in your memory.
By combining these ideas with the previous observation that scaling either vector scales the dot
product by the same factor, we arrive at the general relationship between the dot product and the
cosine.
Dot Product Relation to Intercepted Angle
The dot product of two vectors
$\mathbf{a}$
and
$\mathbf{b}$
is equal to the cosine of the
angle
$\theta $
between the vectors, multiplied by the lengths of the vectors (see
Figure 2.26). Stated formally,
$$\begin{array}{}\text{(2.4)}& \mathbf{a}\cdot \mathbf{b}=\parallel \mathbf{a}\parallel \parallel \mathbf{b}\parallel \mathrm{cos}\theta .\end{array}$$
What does it mean to measure the angle between two vectors in 3D? Any two vectors will always lie
in a common plane (place them tail to tail to see this), and so we measure the angle in the plane
that contains both vectors. If the vectors are parallel, the plane is not unique, but the angle is
either
$0{}^{\mathrm{o}}$
or
$\pm 180{}^{\mathrm{o}}$
, and it doesn't matter which plane we choose.
The dot product provides a way for us to compute the angle between two vectors. Solving
Equation (2.4) for
$\theta $
,
Using the dot product to compute the angle between two vectors
$$\begin{array}{}\text{(2.5)}& \theta & =\mathrm{arccos}\left(\frac{\mathbf{a}\cdot \mathbf{b}}{\parallel \mathbf{a}\parallel \parallel \mathbf{b}\parallel}\right).\end{array}$$
We can avoid the division in Equation (2.5) if we know that
$\mathbf{a}$
and
$\mathbf{b}$
are unit vectors. In this very common case, the denominator of
Equation (2.5) is trivially 1, and we are left with
Computing the angle between two unit vectors
$$\begin{array}{rlr}\theta & =\mathrm{arccos}(\hat{\mathbf{a}}\cdot \hat{\mathbf{b}})& \text{(assume}\hat{\mathbf{a}}\text{and}\hat{\mathbf{b}}\text{are unit vectors)}.\end{array}$$
If we do not need the exact value of
$\theta $
, and need only a classification of the relative
orientation of
$\mathbf{a}$
and
$\mathbf{b}$
, then we need only the sign of the dot
product. This is the same idea illustrated in Figure 2.18, only now we
can relate it to the angle
$\theta $
, as shown in Table 2.1.
$\mathbf{a}\cdot \mathbf{b}$

$\theta $

Angle is 
$\mathbf{a}$
and
$\mathbf{b}$
are 
$>0$

$0{}^{\mathrm{o}}\le \theta <90{}^{\mathrm{o}}$

acute 
pointing mostly in the same direction 
$0$

$\theta =90{}^{\mathrm{o}}$

right 
perpendicular 
$<0$

$90{}^{\mathrm{o}}<\theta \le 180{}^{\mathrm{o}}$

obtuse 
pointing mostly in the opposite direction 
Table 2.1
The sign of the dot product can be used as a rough classification
of the angle between two vectors.
Since the magnitude of the vectors does not affect the sign of the dot product,
Table 2.1 applies regardless of the lengths of
$\mathbf{a}$
and
$\mathbf{b}$
.
However, notice that if either
$\mathbf{a}$
or
$\mathbf{b}$
is the zero vector, then
$\mathbf{a}\cdot \mathbf{b}=0$
. Thus, when we use the dot product to classify the relationship between two
vectors, the dot product acts as if the zero vector is perpendicular to any other vector. As it
turns out, the cross product behaves differently.
Let's summarize the dot product's geometric properties.

The dot product
$\mathbf{a}\cdot \mathbf{b}$
measures the length of the projection
of
$\mathbf{b}$
onto
$\mathbf{a}$
, multiplied by the length of
$\mathbf{a}$
.

The dot product can be used to measure displacement in a particular direction.

The projection operation is closely related to the cosine function. The dot product
$\mathbf{a}\cdot \mathbf{b}$
also is equal to
$\parallel \mathbf{a}\parallel \parallel \mathbf{b}\parallel \mathrm{cos}\theta $
, where
$\theta $
is the angle between the vectors.
We review the commutative and distributive properties of the dot product at the end of this chapter
along with other algebraic properties of vector operations.
2.12Vector Cross Product
The other vector product, known as the cross product, can be
applied only in 3D. Unlike the dot product, which yields a scalar and is commutative, the vector
cross product yields a 3D vector and is not commutative.
2.12.1Official Linear Algebra Rules
Similar to the dot product, the term “cross” product comes from the symbol used in the notation
$\mathbf{a}\times \mathbf{b}$
. We always write the cross symbol, rather than omitting it as we do
with scalar multiplication. The equation for the cross product is
Cross product
$$\left[\begin{array}{c}{x}_{1}\\ {y}_{1}\\ {z}_{1}\end{array}\right]\times \left[\begin{array}{c}{x}_{2}\\ {y}_{2}\\ {z}_{2}\end{array}\right]=\left[\begin{array}{c}{y}_{1}{z}_{2}{z}_{1}{y}_{2}\\ {z}_{1}{x}_{2}{x}_{1}{z}_{2}\\ {x}_{1}{y}_{2}{y}_{1}{x}_{2}\end{array}\right].$$
For example,
$$\left[\begin{array}{c}1\\ 3\\ 4\end{array}\right]\times \left[\begin{array}{c}2\\ 5\\ 8\end{array}\right]=\left[\begin{array}{c}(3)(8)(4)(5)\\ (4)(2)(1)(8)\\ (1)(5)(3)(2)\end{array}\right]=\left[\begin{array}{c}24(20)\\ 88\\ 56\end{array}\right]=\left[\begin{array}{c}44\\ 0\\ 11\end{array}\right].$$
The cross product enjoys the same level of operator precedence as the dot product: multiplication
occurs before addition and subtraction. When dot product and cross product are used together, the
cross product takes precedence:
$\mathbf{a}\cdot \mathbf{b}\times \mathbf{c}=\mathbf{a}\cdot (\mathbf{b}\times \mathbf{c})$
. Luckily, there's an easy way to remember this: it's the only way
it could work. The dot product returns a scalar, and so
$(\mathbf{a}\cdot \mathbf{b})\times \mathbf{c}$
is undefined, since you cannot take the cross product of a scalar and a vector. The
operation
$\mathbf{a}\cdot (\mathbf{b}\times \mathbf{c})$
is known as the triple product.
We present some special properties of this computation in Section 6.1.
As mentioned earlier, the vector cross product is not commutative. In fact, it is
anticommutative:
$\mathbf{a}\times \mathbf{b}=(\mathbf{b}\times \mathbf{a})$
. The
cross product is not associative, either. In general,
$(\mathbf{a}\times \mathbf{b})\times \mathbf{c}\ne \mathbf{a}\times (\mathbf{b}\times \mathbf{c})$
. More vector algebra laws
concerning the cross product are given in Section 2.13.
2.12.2Geometric Interpretation
The cross product yields a vector that is perpendicular to the original two vectors, as illustrated
in Figure 2.27.
The length of
$\mathbf{a}\times \mathbf{b}$
is equal to the product of the magnitudes of
$\mathbf{a}$
and
$\mathbf{b}$
and the sine of the angle between
$\mathbf{a}$
and
$\mathbf{b}$
:
The magnitude of the cross product is related to the sine of the angle between the vectors
$$\parallel \mathbf{a}\times \mathbf{b}\parallel =\parallel \mathbf{a}\parallel \parallel \mathbf{b}\parallel \mathrm{sin}\theta .$$
As it turns out, this is also equal to the area of the parallelogram formed with two sides
$\mathbf{a}$
and
$\mathbf{b}$
. Let's see if we can verify why this is true by using
Figure 2.28.
First, from planar geometry, we know that the area of the parallelogram is
$bh$
, the product of
the base and the height. (In Figure 2.28, the base is
$b=\parallel \mathbf{b}\parallel $
.) We can verify this rule by “clipping” off a triangle from one end and
moving it to the other end, forming a rectangle, as shown in Figure 2.29.
The area of a rectangle is given by its length and width. In this case, this area is the product
$bh$
. Since the area of the rectangle is equal to the area of the parallelogram, the area of the
parallelogram must also be
$bh$
.
Returning to Figure 2.28, let
$a$
and
$b$
be the lengths of
$\mathbf{a}$
and
$\mathbf{b}$
, respectively, and note that
$\mathrm{sin}\theta =h/a$
. Then
$$\begin{array}{rl}A& =bh\\ & =b(a\mathrm{sin}\theta )\\ & =\parallel \mathbf{a}\parallel \parallel \mathbf{b}\parallel \mathrm{sin}\theta \\ & =\parallel \mathbf{a}\times \mathbf{b}\parallel .\end{array}$$
If
$\mathbf{a}$
and
$\mathbf{b}$
are parallel, or if
$\mathbf{a}$
or
$\mathbf{b}$
is the zero
vector, then
$\mathbf{a}\times \mathbf{b}=\mathbf{0}$
. So the cross product interprets the zero
vector as being parallel to every other vector. Notice that this is different from the dot
product, which interprets the zero vector as being perpendicular to every other vector. (Of
course, it is illdefined to describe the zero vector as being perpendicular or parallel to any
vector, since the zero vector has no direction.)
We have stated that
$\mathbf{a}\times \mathbf{b}$
is perpendicular to
$\mathbf{a}$
and
$\mathbf{b}$
. But there are two directions that are perpendicular to
$\mathbf{a}$
and
$\mathbf{b}$
—which of these two directions does
$\mathbf{a}\times \mathbf{b}$
point? We can
determine the direction of
$\mathbf{a}\times \mathbf{b}$
by placing the tail of
$\mathbf{b}$
at
the head of
$\mathbf{a}$
, and examining whether we make a clockwise or counterclockwise turn from
$\mathbf{a}$
to
$\mathbf{b}$
. In a lefthanded coordinate system,
$\mathbf{a}\times \mathbf{b}$
points towards you if the vectors
$\mathbf{a}$
and
$\mathbf{b}$
make a clockwise turn from your
viewpoint, and away from you if
$\mathbf{a}$
and
$\mathbf{b}$
make a counterclockwise turn. In a
righthanded coordinate system, the exact opposite occurs: if
$\mathbf{a}$
and
$\mathbf{b}$
make a
counterclockwise turn,
$\mathbf{a}\times \mathbf{b}$
points towards you, and if
$\mathbf{a}$
and
$\mathbf{b}$
make a clockwise turn,
$\mathbf{a}\times \mathbf{b}$
points away from you.
Figure 2.30 shows clockwise and counterclockwise turns.
Notice that to make the clockwise or counterclockwise determination, we must align the
head of
$\mathbf{a}$
with the tail of
$\mathbf{b}$
. Compare this to Figure 2.26, where the
tails are touching. The tailtotail alignment shown in Figure 2.26 is the correct way
to position the vectors to measure the angle between them, but to judge whether the turn is
clockwise or counterclockwise, the vectors should be aligned headtotail, as shown in
Figure 2.30.
Let's apply this general rule to the specific case of the cardinal axes. Let
$\hat{\mathbf{x}}$
,
$\hat{\mathbf{y}}$
, and
$\hat{\mathbf{z}}$
be unit vectors that point in the
$+x$
,
$+y$
, and
$+z$
directions,
respectively. The results of taking the cross product of each pair of axes are
Cross product of the cardinal axes
$$\begin{array}{rclrcll}\hat{\mathbf{x}}& \hat{\mathbf{y}}& \hat{\mathbf{z}},& & \hat{\mathbf{y}}& \hat{\mathbf{x}}& \hat{\mathbf{z}},\\ \hat{\mathbf{y}}& \hat{\mathbf{z}}& \hat{\mathbf{x}},& & \hat{\mathbf{z}}& \hat{\mathbf{y}}& \hat{\mathbf{x}},\\ \hat{\mathbf{z}}& \hat{\mathbf{x}}& \hat{\mathbf{y}},& & \hat{\mathbf{x}}& \hat{\mathbf{z}}& \hat{\mathbf{y}}.\end{array}$$
You can also remember which way the cross product points by
using your hand, similar to the way we distinguished between lefthanded and righthanded
coordinate spaces in Section 1.3.3. Since we're using a lefthanded
coordinate space in this book, we'll show how it's done using your left hand. Let's say you have
two vectors,
$\mathbf{a}$
and
$\mathbf{b}$
, and you want to figure out which direction
$\mathbf{a}\times \mathbf{b}$
points. Point your thumb in the direction of
$\mathbf{a}$
, and
your index finger (approximately) in the direction of
$\mathbf{b}$
. If
$\mathbf{a}$
and
$\mathbf{b}$
are pointing in nearly the opposite direction, this may be difficult. Just make sure
that if your thumb points exactly in the direction of
$\mathbf{a}$
; then your index finger is on
the same side of
$\mathbf{a}$
as the vector
$\mathbf{b}$
is. With your fingers in this position,
extend your third finger to be perpendicular to your thumb and index finger, similar to what we
did in Section 1.3.3. Your third finger now points in the direction of
$\mathbf{a}\times \mathbf{b}$
.
Of course, a similar trick works with your right hand for righthanded coordinate spaces.
One of the most important uses of the cross product is to create a vector that is perpendicular to
a plane (see Section 9.5), triangle (Section 9.6), or
polygon (Section 9.7).
2.13Linear Algebra Identities
The Greek philosopher Arcesilaus reportedly said, “Where you find the laws most numerous, there
you will find also the greatest injustice.” Well, nobody said vector algebra was fair.
Table 2.2 lists some vector algebra laws that are
occasionally useful but should not be memorized. Several identities are obvious and are listed
for the sake of completeness; all of them can be derived from the definitions given in earlier
sections.
Identity 
Comments 
$\mathbf{a}+\mathbf{b}=\mathbf{b}+\mathbf{a}$

Commutative property of vector addition 
$\mathbf{a}\mathbf{b}=\mathbf{a}+(\mathbf{b})$

Definition of vector subtraction 
$(\mathbf{a}+\mathbf{b})+\mathbf{c}=\mathbf{a}+(\mathbf{b}+\mathbf{c})$

Associative property of vector addition 
$s(t\mathbf{a})=(st)\mathbf{a}$

Associative property of scalar multiplication 
$k(\mathbf{a}+\mathbf{b})=k\mathbf{a}+k\mathbf{b}$

Scalar multiplication distributes over vector addition 
$\parallel k\mathbf{a}\parallel =\leftk\right\parallel \mathbf{a}\parallel $

Multiplying a vector by a scalar scales the magnitude by
a factor equal to the absolute value of the scalar

$\parallel \mathbf{a}\parallel \ge 0$

The magnitude of a vector is nonnegative 
${\parallel \mathbf{a}\parallel}^{2}+{\parallel \mathbf{b}\parallel}^{2}={\parallel \mathbf{a}+\mathbf{b}\parallel}^{2}$

The Pythagorean theorem applied to vector addition. 
$\parallel \mathbf{a}\parallel +\parallel \mathbf{b}\parallel \ge \parallel \mathbf{a}+\mathbf{b}\parallel $

Triangle rule of vector addition. (No side can be longer
than the sum of the other two sides.)

$\mathbf{a}\cdot \mathbf{b}=\mathbf{b}\cdot \mathbf{a}$

Commutative property of dot product 
$\parallel \mathbf{a}\parallel =\sqrt{\mathbf{a}\cdot \mathbf{a}}$

Vector magnitude defined using dot product 
$k(\mathbf{a}\cdot \mathbf{b})=(k\mathbf{a})\cdot \mathbf{b}=\mathbf{a}\cdot (k\mathbf{b})$

Associative property of scalar multiplication with dot
product

$\mathbf{a}\cdot (\mathbf{b}+\mathbf{c})=\mathbf{a}\cdot \mathbf{b}+\mathbf{a}\cdot \mathbf{c}$

Dot product distributes over vector addition and
subtraction

$\mathbf{a}\times \mathbf{a}=\mathbf{0}$

The cross product of any vector with itself is the zero
vector. (Because any vector is parallel with itself.)

$\mathbf{a}\times \mathbf{b}=(\mathbf{b}\times \mathbf{a})$

Cross product is anticommutative. 
$\mathbf{a}\times \mathbf{b}=(\mathbf{a})\times (\mathbf{b})$

Negating both operands to the cross product results in
the same vector.

$k(\mathbf{a}\times \mathbf{b})=(k\mathbf{a})\times \mathbf{b}=\mathbf{a}\times (k\mathbf{b})$

Associative property of scalar multiplication with cross
product.

$\mathbf{a}\times (\mathbf{b}+\mathbf{c})=\mathbf{a}\times \mathbf{b}+\mathbf{a}\times \mathbf{c}$

Cross product distributes over vector addition and
subtraction.

Table 2.2Table of vector algebra identities
Exercises

Let
$$\mathbf{a}=\left[\begin{array}{cc}3& 8\end{array}\right],{\textstyle \phantom{\rule{1em}{0ex}}}\mathbf{b}=\left[\begin{array}{c}4\\ 0\\ 5\end{array}\right],{\textstyle \phantom{\rule{1em}{0ex}}}\mathbf{c}=\left[\begin{array}{c}16\\ 1\\ 4\\ 6\end{array}\right].$$

Identify
$\mathbf{a}$
,
$\mathbf{b}$
, and
$\mathbf{c}$
, as row or
column vectors, and give the
dimension of each vector.

Compute
${b}_{y}+{c}_{w}+{a}_{x}+{b}_{z}$
.

Identify the quantities in each of the following sentences as scalar or vector.
For vector quantities, give the magnitude and direction. (Note: some directions
may be implicit.)

How much do you weigh?

Do you have any idea how fast you were going?

It's two blocks north of here.

We're cruising from Los Angeles to New York at 600 mph, at an altitude of
33,000 ft.

Give the values of the following vectors. The darker grid lines represent one unit.

Identify the following statements as true or false. If the statement is
false, explain why.

The size of a vector in a diagram doesn't matter; we just need to
draw it in the right place.

The displacement expressed by a vector can be visualized as a
sequence of axially aligned displacements.

These axially aligned displacements from the previous question
must occur in order.

The vector
$[x,y]$
gives the displacement from the point
$(x,y)$
to the origin.

Evaluate the following vector expressions:

$\left[\begin{array}{cc}3& 7\end{array}\right]$

$\parallel \left[\begin{array}{cc}12& 5\end{array}\right]\parallel $

$\parallel \left[\begin{array}{ccc}8& 3& 1/2\end{array}\right]\parallel $

$3\left[\begin{array}{ccc}4& 7& 0\end{array}\right]$

$\left[\begin{array}{cc}4& 5\end{array}\right]/2$

Normalize the following vectors:

$\left[\begin{array}{cc}12& 5\end{array}\right]$

$\left[\begin{array}{cc}0& 743.632\end{array}\right]$

$\left[\begin{array}{ccc}8& 3& 1/2\end{array}\right]$

$\left[\begin{array}{ccc}12& 3& 4\end{array}\right]$

$\left[\begin{array}{cccc}1& 1& 1& 1\end{array}\right]$

Evaluate the following vector expressions:

$\left[\begin{array}{ccc}7& 2& 3\end{array}\right]+\left[\begin{array}{ccc}6& 6& 4\end{array}\right]$

$\left[\begin{array}{ccc}2& 9& 1\end{array}\right]+\left[\begin{array}{ccc}2& 9& 1\end{array}\right]$

$\left[\begin{array}{c}3\\ 10\\ 7\end{array}\right]\left[\begin{array}{c}8\\ 7\\ 4\end{array}\right]$

$\left[\begin{array}{c}4\\ 5\\ 11\end{array}\right]\left[\begin{array}{c}4\\ 5\\ 11\end{array}\right]$

$3\left[\begin{array}{c}a\\ b\\ c\end{array}\right]4\left[\begin{array}{c}2\\ 10\\ 6\end{array}\right]$

Compute the distance between the following pairs of points:

$\left[\begin{array}{c}10\\ 6\end{array}\right],\left[\begin{array}{c}14\\ 30\end{array}\right]$

$\left[\begin{array}{c}0\\ 0\end{array}\right],\left[\begin{array}{c}12\\ 5\end{array}\right]$

$\left[\begin{array}{c}3\\ 10\\ 7\end{array}\right],\left[\begin{array}{c}8\\ 7\\ 4\end{array}\right]$

$\left[\begin{array}{c}2\\ 4\\ 9\end{array}\right],\left[\begin{array}{c}6\\ 7\\ 9.5\end{array}\right]$

$\left[\begin{array}{c}4\\ 4\\ 4\\ 4\end{array}\right],\left[\begin{array}{c}6\\ 6\\ 6\\ 6\end{array}\right]$

Evaluate the following vector expressions:

$\left[\begin{array}{c}2\\ 6\end{array}\right]\cdot \left[\begin{array}{c}3\\ 8\end{array}\right]$

$7\left[\begin{array}{cc}1& 2\end{array}\right]\cdot \left[\begin{array}{cc}11& 4\end{array}\right]$

$10+\left[\begin{array}{c}5\\ 1\\ 3\end{array}\right]\cdot \left[\begin{array}{c}4\\ 13\\ 9\end{array}\right]$

$3\left[\begin{array}{c}2\\ 0\\ 4\end{array}\right]\cdot (\left[\begin{array}{c}8\\ 2\\ 3/2\end{array}\right]+\left[\begin{array}{c}0\\ 9\\ 7\end{array}\right])$

Given the two vectors
$$\mathbf{v}=\left[\begin{array}{c}4\\ 3\\ 1\end{array}\right],{\textstyle \phantom{\rule{1em}{0ex}}}\hat{\mathbf{n}}=\left[\begin{array}{c}\sqrt{2}/2\\ \sqrt{2}/2\\ 0\end{array}\right],$$
separate
$\mathbf{v}$
into components that are perpendicular
and parallel to
$\hat{\mathbf{n}}$
. (As the notation implies,
$\hat{\mathbf{n}}$
is a unit
vector.)

Use the geometric definition of the dot product
$$\begin{array}{r}\mathbf{a}\cdot \mathbf{b}=\parallel \mathbf{a}\parallel \parallel \mathbf{b}\parallel \mathrm{cos}\theta \end{array}$$
to prove the law of cosines.

Use trigonometric identities and the algebraic definition of the dot product in 2D
$$\begin{array}{r}\mathbf{a}\cdot \mathbf{b}={a}_{x}{b}_{x}+{a}_{y}{b}_{y}\end{array}$$
to prove the geometric interpretation of the dot product in 2D. (Hint: draw a diagram of the vectors and all angles involved.)

Calculate
$\mathbf{a}\times \mathbf{b}$
and
$\mathbf{b}\times \mathbf{a}$
for the following vectors:

$\mathbf{a}=\left[\begin{array}{ccc}0& 1& 0\end{array}\right],\mathbf{b}=\left[\begin{array}{ccc}0& 0& 1\end{array}\right]$

$\mathbf{a}=\left[\begin{array}{ccc}2& 4& 1\end{array}\right],\mathbf{b}=\left[\begin{array}{ccc}1& 2& 1\end{array}\right]$

$\mathbf{a}=\left[\begin{array}{ccc}3& 10& 7\end{array}\right],\mathbf{b}=\left[\begin{array}{ccc}8& 7& 4\end{array}\right]$

Prove the equation for the magnitude of the cross product
$$\begin{array}{r}\parallel \mathbf{a}\times \mathbf{b}\parallel =\parallel \mathbf{a}\parallel \parallel \mathbf{b}\parallel \mathrm{sin}\theta .\end{array}$$
(Hint: make use of the geometric interpretation of the dot product and
try to show how the left and right sides of the equation are equivalent,
rather than trying to derive one side from the other.)

Section 2.8 introduced the norm of a vector, namely, a scalar value associated with a given vector.
However, the definition of the norm given in that section is not the only definition of a norm for a vector. In general,
the
$p$
norm of an
$n$
dimensional vector is defined as
$${\parallel \mathbf{x}\parallel}_{p}\equiv {\left(\sum _{i=1}^{n}{\left{x}_{i}\right}^{p}\right)}^{1/p}.$$
Some of the more common
$p$
norms include:

The
${L}^{1}$
norm, a.k.a. Taxicab norm (
$p=1$
):
$${\parallel \mathbf{x}\parallel}_{1}\equiv \sum _{i=1}^{n}\left{x}_{i}\right.$$

The
${L}^{2}$
norm, a.k.a. Euclidean norm (
$p=2$
). This is the most common and
familiar norm, since it measures geometric length:
$${\parallel \mathbf{x}\parallel}_{2}\equiv \sqrt{\sum _{i=1}^{n}{x}_{i}^{2}}.$$

The infinity norm, a.k.a. Chebyshev norm (
$p=\mathrm{\infty}$
):
$${\parallel \mathbf{x}\parallel}_{\mathrm{\infty}}\equiv max(\left{x}_{1}\right,\dots ,\left{x}_{n}\right).$$
Each of these norms can be thought of as a way to assigning a length or size to a vector.
The Euclidean norm was discussed in Section 2.8.
The Taxicab norm gets its name from how a taxicab would measure distance driving the streets of a city laid out in a grid (e.g., Cartesia from Section 1.2.1).
For example, a taxicab that drives 1 block east and 1 block north drives a total distance of 2 blocks, whereas a bird flying “as the crow flies” can fly in a straight
line from start to finish and travel only
$\sqrt{2}$
blocks (Euclidean norm).
The Chebyshev norm is simply the absolute value of the vector component with the largest absolute value. An example of how this norm can be used is to consider the number
of moves required to move a king in a game of chess from one square to another. The immediately surrounding squares require 1 move, the squares surrounding those require
2 moves, and so on.

(a)For each of the following find
${\parallel \mathbf{x}\parallel}_{1}$
,
${\parallel \mathbf{x}\parallel}_{2}$
,
${\parallel \mathbf{x}\parallel}_{3}$
, and
${\parallel \mathbf{x}\parallel}_{\mathrm{\infty}}$
:
 (1)
$\left[\begin{array}{cc}3& 4\end{array}\right]$
 (2)
$\left[\begin{array}{cc}5& 12\end{array}\right]$
 (3)
$\left[\begin{array}{ccc}2& 10& 7\end{array}\right]$
 (4)
$\left[\begin{array}{ccc}6& 1& 9\end{array}\right]$
 (5)
$\left[\begin{array}{cccc}2& 2& 2& 2\end{array}\right]$

*(b)Draw the unit circle (i.e., the set of all vectors with
${\parallel \mathbf{x}\parallel}_{p}=1$
) centered at the origin for the
${L}^{1}$
norm,
${L}^{2}$
norm, and infinity norm.

A man is boarding a plane. The airline has a rule that
no carryon item may be more than two feet long, two feet
wide, or two feet tall. He has a very valuable sword that
is three feet long, yet he is able to carry the sword on board
with him.
How is he able to do this? What is the longest
possible item that he could carry on?

Verify Figure 2.11 numerically.

Is the coordinate system used in Figure 2.27 a lefthanded or righthanded
coordinate system?

One common way of defining a bounding box for a 2D object is to specify a
center point
$\mathbf{c}$
and a radius vector
$\mathbf{r}$
, where each
component of
$\mathbf{r}$
is half the length of the side of the bounding box
along the corresponding axis.

Describe the four corners
${\mathbf{p}}_{\text{UpperLeft}}$
,
${\mathbf{p}}_{\text{UpperRight}}$
,
${\mathbf{p}}_{\text{LowerLeft}}$
, and
${\mathbf{p}}_{\text{LowerRight}}$
.

Describe the eight corners of a bounding cube, extending this idea into 3D.

A nonplayer character (NPC) is standing at location
$\mathbf{p}$
with a forward direction of
$\mathbf{v}$
.

How can the dot product be used to determine whether the point
$\mathbf{x}$
is in front of or behind the NPC?

Let
$\mathbf{p}=\left[\begin{array}{cc}3& 4\end{array}\right]$
and
$\mathbf{v}=\left[\begin{array}{cc}5& 2\end{array}\right]$
.
For each of the following points
$\mathbf{x}$
determine whether
$\mathbf{x}$
is in front of or behind the NPC:
 (1)
$\mathbf{x}=\left[\begin{array}{cc}0& 0\end{array}\right]$
 (2)
$\mathbf{x}=\left[\begin{array}{cc}1& 6\end{array}\right]$
 (3)
$\mathbf{x}=\left[\begin{array}{cc}6& 0\end{array}\right]$
 (4)
$\mathbf{x}=\left[\begin{array}{cc}4& 7\end{array}\right]$
 (5)
$\mathbf{x}=\left[\begin{array}{cc}5& 5\end{array}\right]$
 (6)
$\mathbf{x}=\left[\begin{array}{cc}3& 0\end{array}\right]$
 (7)
$\mathbf{x}=\left[\begin{array}{cc}6& 3.5\end{array}\right]$

Extending the concept from Exercise 20, consider the case where the NPC
has a limited field of view (FOV). If the total FOV angle is
$\varphi $
, then the
NPC can see to the left or right of its forward direction by a maximum angle of
$\varphi /2$
.

How can the dot product be used to determine whether the point
$\mathbf{x}$
is visible to the NPC?

For each of the points
$\mathbf{x}$
in Exercise 20
determine whether
$\mathbf{x}$
is visible to the NPC if its FOV is
${90}^{\circ}$
.

Suppose that the NPC's viewing distance is also limited to a maximum
distance of 7 units. Which points are visible to the NPC then?

Consider three points labeled
$\mathbf{a}$
,
$\mathbf{b}$
, and
$\mathbf{c}$
in the
$xz$
plane of our lefthanded coordinate system, which represent
waypoints on an NPC's path.

How can the cross product be used to determine whether, when moving from
$\mathbf{a}$
to
$\mathbf{b}$
to
$\mathbf{c}$
, the NPC makes a clockwise or
counterclockwise turn at
$\mathbf{b}$
, when viewing the path from above?

For each of the following sets of three points, determine whether the NPC
is turning clockwise or counterclockwise when moving from
$\mathbf{a}$
to
$\mathbf{b}$
to
$\mathbf{c}$
:

(1)
$\mathbf{a}=\left[\begin{array}{ccc}2& 0& 3\end{array}\right]$
,
$\mathbf{b}=\left[\begin{array}{ccc}1& 0& 5\end{array}\right]$
,
$\mathbf{c}=\left[\begin{array}{ccc}4& 0& 1\end{array}\right]$

(2)
$\mathbf{a}=\left[\begin{array}{ccc}3& 0& 5\end{array}\right]$
,
$\mathbf{b}=\left[\begin{array}{ccc}4& 0& 0\end{array}\right]$
,
$\mathbf{c}=\left[\begin{array}{ccc}3& 0& 3\end{array}\right]$

(3)
$\mathbf{a}=\left[\begin{array}{ccc}1& 0& 4\end{array}\right]$
,
$\mathbf{b}=\left[\begin{array}{ccc}7& 0& 1\end{array}\right]$
,
$\mathbf{c}=\left[\begin{array}{ccc}5& 0& 6\end{array}\right]$

(4)
$\mathbf{a}=\left[\begin{array}{ccc}2& 0& 1\end{array}\right]$
,
$\mathbf{b}=\left[\begin{array}{ccc}1& 0& 2\end{array}\right]$
,
$\mathbf{c}=\left[\begin{array}{ccc}4& 0& 4\end{array}\right]$

In the derivation of a matrix to scale along an
arbitrary axis, we reach a step where we have the vector expression
$${\mathbf{p}}^{\prime}=\mathbf{p}+(k1)(\mathbf{p}\cdot \mathbf{n})\mathbf{n},$$
where
$\mathbf{n}$
is an arbitrary vector
$[{n}_{x},{n}_{y},{n}_{z}]$
and
$k$
is an arbitrary scalar, but
$\mathbf{p}$
is one of the cardinal axes. Plug
in the value
$\mathbf{p}=[1,0,0]$
and simplify the resulting expression for
${\mathbf{p}}^{\prime}$
.
The answer is not a vector expression, but a single vector, where the scalar expressions
for each coordinate have been simplified.

A similar problem arises with the derivation of a matrix to rotate about an arbitrary
axis. Given an arbitrary scalar
$\theta $
and a vector
$\mathbf{n}$
, substitute
$\mathbf{p}=[1,0,0]$
and simplify the value of
${\mathbf{p}}^{\prime}$
in the expression
$${\mathbf{p}}^{\prime}=\mathrm{cos}\theta (\mathbf{p}(\mathbf{p}\cdot \mathbf{n})\mathbf{n})+\mathrm{sin}\theta (\mathbf{n}\times \mathbf{p})+(\mathbf{p}\cdot \mathbf{n})\mathbf{n}.$$
What's our vector, Victor?
— Captain Oveur in Airplane! (1980)
References
[1]
Tevian Dray and Corinne A. Manogue.
“The Geometry of the Dot and Cross Products.”
Journal of Online Mathematics and Its Applications 6.
http://mathdl.maa.org/images/upload_library/4/vol6/Dray2/Dray.pdf.
[2]
Fletcher Dunn and Ian Parberry.
3D Math Primer for Graphics and Game Development, First
edition.
Plano, TX: Wordware Publishing, 2002.