Vectors, Vectors Everywhere

Vectors, Vectors Everywhere... by Brandon Williams
Main Index...

I was planning on writing a small tutorial on vectors which was to be included in my last 3D tutorial, but this got so lengthy I decided to put it entirely separate.

I would not be lying if I told you that vectors can/will/are going to be the most important things you can learn. And the sad thing about that is that I am fairly new to them, but I am going to try my best to sum up everything I have learned about them in the past few weeks. The prerequisite for this tutorial are not as many as some of my other tutorials. I suggest a pretty good understanding of Trigonometry. You are going to need a lot of practice with vectors because they are somewhat difficult. Also there is a lot of terminology that you are going to have to become familiar with. I will put all the important words in bold. Let's get started...

What is a vector?
Surprisingly enough this seems to be the hardest part for people to understand. The actual concept of vectors. Now do not take my explanation of them as if I am being ambiguous or anything. Take it as literal as possible. A vector is merely a quantity that has both magnitude and direction. Vectors are best represented as arrows. Here is your most basic vector:

Some terms for you to learn. Those vector have certain components that you need to learn. The head of a vector is the end with the arrow. The tail is at the other end...its where the vector starts. The magnitude is the length of the vector. The direction is the angle the vector makes with a horizontal line going through the tail.

You have two vectors in that picture: vector v and vector u. When printed vectors are usually denoted with bold face lettering. Whenever you do work on paper you denote a vector with an arrow above the letter.

Those two vectors have equal magnitude but opposite direction. You could express u in terms of v like this: u = -v. So a vector multiplied by negative one has an opposite direction. The norm of v, written as ||v||, is the magnitude or length of vector v. If two vectors have the same magnitude and same direction then they are the same vector and called equivalent. To find the norm of a vector you use the Pythagorean Theorem. You need to know the coordinate of the tail and the head to do this though.

Now some arithmetic with vectors. There are two ways to add vectors. If the two vectors share the same initial point then you can make a parallelogram and connect the diagonal. I know that is a weak explanation so I made a quick Flash movie:

So adding vectors is pretty easy. Subtracting is pretty much the same. We can actually put subtraction in terms of addition (if you remember Algebra): v - u = v + (-u). So now all you do is flip u so that it is negative (opposite direction) and then add them like normal.

But there is still one more way which is a little easier. The only reason I told you about the above way is because we are going to use it the most later on. Here is a Flash movie for the second way:

That is a little easier. And subtraction would be the same except you flip the vector you are subtracting. Now here is another term for you. The new vector that we came up with when we were adding and subtracting vectors is called the resultant.

The best thing for you to do now would be to practice with a few problems and test yourself on some terminology. By the end of this we are going to be moving very fast.

We have talked about vectors, which were quantities with magnitude and direction, but now we are going to learn about scalars, which are quantities with only a magnitude value. You multiply vectors and scalars. The product can be a number of things. It could result in the lengthening of the vector or the shortening, or the vector may go in the opposite direction, or a combination of the two. Let's call s a scalar (I will use italics to denote a scalar), here are some properties that one can observe when multiplying scalars and vectors:

      Consider:  sv = u

           u will be longer than v if |s| > 1
           u will be shorter than v if |s| < 1
           u will go in the opposite direction of v if s < 0
           In general:  ||sv|| = |s| ||v||

Do not worry if any of that went over your head because we will discuss it throughout the rest of this. Also I'm sure that if you stared at it long enough it would come to you...just give it time. Here are two pictures that may help you understand scalars:

Scalars are really that simple. Nothing abstract at all.

Before we move on I thought I would give a few sample problems just to make sure you know everything so far. I am going to make some pictures out of ASCII characters because I do not feel like making another picture but hopefully you can get what I am trying to say.

Give vector v, draw and find the norm of each each vector:

        |         . (3,4)
        |        /
        |   v  /
        |    /
        |  /
        |/____________
      (0,0)

   a.) 2v    b.) .5v    c.) -4v

I'm not going to put up the answers because I want you to figure it out but if you are having trouble email me.

Next we need to know how to break a vector into its x- and y-components. I actually discussed this in my "Math Tricks" so I am going to be kind of brief here. First some terms. When you separate a vector into parts you are basically finding what two vector's sum will equal the original vector. Take for instance vector v from the above problem. Couldn't that be expressed in terms of a vector w with a magnitude of three going directly right and vector u with a magnitude of four going directly up? The sum of w and u would equal v. Just try it out yourself...it should be easy to see without having to write anything down if you think of connecting the diagonals of a parallelogram. Well those two vectors, w and u (those are not official names, just letters I used), are called component vectors (that is an official name however).

So if you were given a vector with a magnitude of r and a direction of q you could use this to break it down to its component vectors:

x = r cos q
y = r sin q

Now let's talk about why. Its actually very simple but I have made a promise to myself that I am going to prove everything I talk about because if you are like me then you hate being given information without knowing how/why. First remember some good ol' fashion Trigonometry:

              /|
        r   /  |
          /    | y
        /      |
      /_t______|
          x

      cos t = x / r
      sin t = y / r

      -- solve for x and y by multiplying both sides by r --

      r cos t = x
      r sin t = y

      -- and there you go...not very hard --

So thats how to break a vector into its component vectors. If you are still unsure of the "component vectors" then just think of it as a line in terms of length along the x-axis and length along the y-axis.

It is now time for the harder stuff. This next part (its the middle...everything at the beginning and end is easy) will require the most from you. I am first going to introduce the actual notation for a vector. We have been using boldface letters hitherto but that does not mean much. I am also going to introduce a few other things in the same paragraph...look at this:

      Notation for vector v
            v = (x, y)  -- used to denote a vector with its tail
                           at (0,0) and its head at (x,y)

      The coordinates x and y are now the x- and y-components.

      The magnitude or norm of v is
                      ________
            ||v|| = \/ x² + y²

Easy thus far. That is probably even more straight forward then what I did at first. Now the unit vector. A unit vector is simply a vector with a magnitude of 1. There are two special unit vectors, however, that get their own special name. The unit vector on the horizontal is denoted with an i and the unit vector on the vertical is denoted with a j. Even easier to remember: i = (1,0) and j = (0,1). There is actually on more for 3D which is k = (0,0,1) but we are not going to be dealing with that for right now. Before I go more into unit vectors let me show you what vector arithmetic looks like with our new notation:

      Given vectors v = (x, y) and w = (a, b)

      Vector Sum
            v + w = (x,y) + (a,b)
                  = (x+a, y+b)

      Vector Difference
            v - w = (x,y) - (a,b)
                  = (x-a, y-b)

      Scalar Multiplication
            sv = s(x,y) = (sx, sy)

Also there is a way of defining a direction vector with two given points. When you find a direction vector you are just find the vector (Ex: v = [x,y]) with two given points. Its kind of like connecting the two points with a line (a vector) and then sliding it down the x- and y-axis so that the tail is at (0,0). An example: say you were given point A(a,b) and point B(c,d), you could find the direction vector by simply subtracting the second one from the first: d = (c-a, d-b).

Like a lot of things in mathematics there are tons of different ways to write things. In the case of vectors we are going to express a vector as the sum of scalar multiples of the unit vectors i and j.

      Think of the vector v = (x,y)
            (x,y) = x(1,0) + y(0,1) = xi + yj

      (x,y) = xi + yj   <-- Component Form

      -- That is easy...the same can be done for the
         vectors in polar form
            (x,y) = r cosq (1,0) + r sinq (0,1) = r cosq i + r sinq j

      (x,y) = r cosq i + r sinq j   <-- Polar Form

You can also express a unit vector in the same direction of another vector v with this equation:

           v
     u = -----
         ||v||

All of the above equations work even when the head and tail of the vector are at other points other than (0,0). The easiest way to do this is get the vector in a notation we are familiar with. For example, if we knew that a vector with an initial point of (1,2) and the head at (4,3) we could simple slide the vector down the x- and y-axis to get the tail at (0,0). How do you do that? Simply subtract the coordinates of the tail from the head: (4,3) - (1,2) = (4-1, 3-2) = (3,1). That coordinate, (3,1), is a vector now at (0,0). Now putting it into the various forms should be pretty easy. That is what I was talking about when I mentioned the direction vector earlier. Here are a few examples..I know this part is a little hard so I'll work the problem with you...but please try them yourself too:

      Given v with an initial point of (3,7) and a terminal point of (8,2)
      determine the...

            a.) Component Form

            b.) Polar Form

            c.) Unit vector in the same direction as v


      Do not look past this point until you have tried to figure it
      out yourself.  If you do not try then you are only hurting yourself.

      --------------------------------------------------------------------

      Answers:

           a.) (8-3)i + (2-7)j = 5i - 5j

                                        _____________     __________     _
           b.) -- First find "r": r = \/(8-3)²+(2-7)² = \/(5)²+(-5)² = 5\/2
               -- Find q: tanq=-5/5  -> q=tan^-1(-1)  -> q=315
                              _               _
               -- (5,-5) = 5\/2 cos 315 i + 5\/2 sin 315 j

           c.)      (8-3, 7-2)
               u = ----------- = [25*sqrt(2), 25*sqrt(2)]
                    5*sqrt(2)

               -- you can check (c) above by using the Pythagorean Theorem
                  on (x,y) to see if it equals one.

Please, please look at that for a long time if you do not get it...you have come to far now to turn back! I know its confusing stuff but just stick with me for a second...the cool stuff is coming up.

Next we would discuss parametric equations and vectors but I'm skipping...I think you need a break. I cannot remember if we are going to need for whats coming up (I'm kind of writing this as I think of it), but if it turns out that I skipped something I will go over it. Parametric equations and vectors are two things I have a hard time with.

Earlier we multiplied a scalar by a vector right? Something to the extent of: v = (x,y), s = scalar,
sv = (sx, sy)...right? Well it is also possible to find the product of two vectors. It is called the dot product. An important thing to know about the dot product is that it results in a scalar...not another vector. Here is the equation:

      If v = (v1, v2) and w = (w1, w2), then
            v . w = v1*w1 + v2*w2

Note that the boldface period, ".", represents the dot product. So as you can see from that equation that you will come out with a scalar. That may not seem too useful right now, and it may be kind of hard to imagine what that scalar looks like but it should come together in a minute.

First, let's just examine something that is going to seem kind of random. What would the dot product of the same vector be? Let's find out:

      v = (v1, v2)
      v . v = v1*v1+v2*v2
            = v1² + v2²
            = ||v||²

Do you see that relationship? The dot product of a vector with itself is its magnitude squared. We'll use this in a minute. For now let's look at some other ways of writing the dot product. Here is a simple picture that shows vector v and w and the resultant of w minus v:

Hopefully you can look at that and see exactly what is going on. If not then just imagine v flipped in the opposite direction (since it is a subtraction problem) and then connect v's tail with w's head. Now you see it? Let's look at this situation in a different way :

      -- Now if we are to assume that v and w are perpendicular then
         the Pythagorean Theorem would tell use that:
            ||w - v||² = ||w||² + ||v||²

      -- But if they are not perpendicular then we can do this:

      If w = (w₁, w₂) and v = (v₁, v₂)
            -- this line is obvious --
            ||w - v||² = ||(w₁-v₁, w₂-v₂)||²

            -- use the dot product on the right side --
                       = (w₁-v₁)² + (w₂-v₂)²

            -- expand the binomials using F.O.I.L. --
                       = (w₁² - 2w₁v₁ + v₁²) + (w₂² - 2w₂v₂ + v₂²)

            -- group W's and V's and organize --
                       = (w₁² + w₂²) + (v₁² + v₂²) - 2(w₁v₁ + w₂v₂)

            -- remember that a vector squared equals its magnitude
               squared...I also substituted the last term for the dot
               product
            ||w - v||² = ||w||² + ||v||² - 2w . v

Now I'm going to stop right there for a second to point something out that is very important...hopefully you are following along. Now when we started this we could express (||w - v||) with the Pythagorean Theorem if and only if the two vectors were perpendicular. Then just now we came up with a somewhat long way of finding (||w - v||) without the vectors being perpendicular. So let me ask you...what is the difference between the very first equation and that last one just above? Its the (-2w . v) right? So now one can extrapolate our given information to the fact that when the vectors are perpendicular (-2w . v = 0). Extrapolate even further and we can just say that
(w . v = 0)...its the dot product of w and v isn't it? So something that may come up a lot...if the dot product of two vectors is zero then those two vectors are perpendicular. If you want some terminology thrown into the mix then you can call perpendicular vectors orthogonal vectors.

Whew! I'm almost out of breath...but still so much to go over and we are still on just the dot product. If you read my "3d Tutorial" then you would have seen my derivation of the Law of Cosines. If not then you can either go read it here (about midway) or check out this picture of the proof done on paper (by me...sorry for the bad handwriting). If you look at the Law of Cosines and that last equation just above they look similar right? Here is how to express the Law of Cosines with vectors...first a picture without vectors:

For a picture like that the Law of Cosines would state:

      a² = b² + c² - (2*a*b)cos q

So if you were given two sides of a triangle an the angle between them you could find the other side with the above equation. Now let's look at a picture that is pretty much the same except expressed with vectors:

There is really not all that much different from that picture and the other one. The Law of Cosines for a picture like that would be:

      ||w-v||² = ||v||² + ||w||² - (2||v|| * ||v||)cos q

      -- Now we can substitue some things from what we learned
         earlier.  Remember this? :

         ||w - v||² = ||w||² + ||v||² - 2w . v

      -- Substitute the right side of the above equation for the left
         side of the one above that since that are equal to each other:

       ||w||² + ||v||² - 2w . v = ||v||² + ||w||² - (2||v|| * ||v||)cos q

      -- Now you clean it up a little.  First of all you have a:
         ||w||² + ||v||² , on both sides of the equation so they
         cancel out and you are left with:

         - 2w . v = - (2||v|| * ||v||)cos q

      -- Then divide both sides by [-(2||v|| * ||v||)] and you get this:

               -2w . v
         ------------------ = cos q
          -(2||v|| * ||v||)

      -- Simplify by canceling out the -2's in the numerator and
         denominator of the fraction on the left:

              w . v
         -------------- = cos q
          ||w|| * ||v||

And now we are done with dot products. That last equation is something you will see a lot and is big in computer graphics. Its used for polygon shading and hidden surface removal (geometry culling). Basically you can find the angle of two vectors by taking the inverted cosine of the dot product of w and v divided by the product of the magnitudes of the two vectors. A lot of times you do not even need to use the inverted cosine function...the ratio is usually good enough.

I am going to give you a few properties of the dot product but I am going to leave it to you to see why they are. Its very easy actually...just plug-in a few numbers and see that they work...thats all:

      v . v = ||v||²                      Norm

          -- note: that basically says that the dot product of the same
             vector results in the vector's norm squared.

      v . w = w . v                       Commutative

      u . (v + w) = u . v + u . w         Distributive

      (ru) . v = r(u . v)                 Associative

      ||rw|| = (|r|)(||w||)               Scalar

Although those are good to know and will help you later it is not crucial that you memorize them. Most of them are common sense anyway.

Now hitherto we have been dealing with vectors in 2D space. The great thing about discussing vectors in 3D space is that most of the equations are almost identical. The notation of a vector is pretty much the same: v = (x, y, z) except now you have a ordered triplet defining the head of the vector...the tail is still at (0, 0, 0). Here are the other vector relationships in 3D that are similar to 2D:

      v = (v1, v2, v3)  and  w = (w1, w2, w3)
                ________________
      ||v|| = \/ v1² + v2² + v3²

      v + w = (v1+w1, v2+w2, v3+w3)

      v + w = (v1-w1, v2-w2, v3-w3)

      av = a(v1, v2, v3) = (a*v1, a*v2, a*v3)

      v . w = (v1, v2, v3) * (w1, w2, w3) = v1*w1 + v2*w2 + v3*w3

      -- And the unit vector in the same direction:

            v1        v2        v3
      u = ----- i + ----- j + ----- k
          ||v||     ||v||     ||v||

          -- where i = (1,0,0) , j = (0,1,0) , and k = (0,0,1)

As you can see all that was just an extension of 2D. You can also use the same equation for finding the angle between two vectors in 3D. Something that is used all the time in 3D games is the cross product. The cross product is only for 3D vectors...thats the reason we never discussed it when we were talking 2D. You use the cross product on two vectors and its result is another vector perpendicular to the other two. The is the equations:

      v = (v1, v2, v3)  and  w = (w1, w2, w3)

      v x w = (v2*w3 - v3*w2, v3*w1 - v1*w3, v1*w2 - v2*w1)

The x denotes a cross product...not multiplication. Now that equation is definitely not easy to remember. You might want to write down so you will have it to look at whenever you need to. The derivation of the cross product is somewhat out of our scope for right now. Just in case you are curious it is the result of expanding a 3x3 determinant. Matrices and determinants are something I have hardly even touched on and I'm not about to start in this tutorial. If anything I could devote three or four tutorials to only matrices. Matrices and vectors are usually taught together in a class called Linear Algebra.

Now let's really think in three dimensions...in particular a plane. In mathematics a plane in the n^th dimension is a surface with n - 1 dimensions. Meaning in 3D a plane is a two-dimensional flat surface...in 2D a plane is a one-dimensional flat surface. I am going to denote a plane with an italicized capital letter. The equation for a plane in 3D is: ax + by + cz = d , where "d" is a constant and (a,b,c) is a vector perpendicular to the plane.

More terminology: a vector perpendicular to a plane is called the normal vector.

Back to the plane stuff. Finding the equation of a plane is pretty important in various applications ...I'm not going to discuss them right now, but you will probably see in a second. Suppose you had a point, P, in the plane H, and a vector, n, that had its tail at point P. Vector n is perpendicular to the plane if it is perpendicular to every other vector in plane H that contains P. Here is a picture:

A brief description of the picture: you have plane H with two arbitrary points, P and Q, on the plane. Vector v is the vector with its tail at P and its head at Q and it lies on the plane. Vector n is perpendicular to the plane, and vector v. To find vector v is the same as finding the direction vector...we learned that towards the beginning of this tutorial. Just to refresh you if you are given two points you can find the direction vector by subtracting the second point from the first. So to find v you would subtract Q from P. Then you could find the dot product of v and n and if it equals zero then you know they are perpendicular right? I hope you remember that from above. Well here is a generalized equation for what I am talking about:

      // Given point Q and P on a plane and vector n which is normal
         to the plane and has P as its head this will hold true

         n . (Q - P) = 0

Now the thing that is so great about that equation is that you can use it to find the equation of a plane. Say you knew the normal vector of a plane as well as a point on the plane, you could use the above to solve for ax + by + cz = d , form. Here is a sample problem:

      Determine the equation of a plane containing the point P(2,2,3)
      with normal vector (3,6,-1).


      -- first set up the information you have: n=(3,6,-1), P=(2,2,3),
         and Q=(x,y,z).  Q is the unknown point on the plane:

         (3,6,-1) . [(x,y,z) - (2,2,3)] = 0

         (3,6,-1) . (x-2, y-2, z-3) = 0

         3(x-2) + 6(y-2) + -1(z-3) = 0

         (3x-6) + (6y-12) + (-z+3) = 0

         3x + 6y - z = 6 + 12 - 3

         3x + 6y - z = 15   <- that is the equation of the plane

Notice any kind of relationship between the plane equation and the normal vector? The plane equation's coefficients are the same as the components of the normal vector. Another relationship that can be seen is that the dot product of the normal vector and P is equal to the constant "d." That may take some thinking to see why. I will show you the work to prove it and comment it as much as I can along the way:

      // let the normal vector, n, equal (n1, n2, n3)
      // let point Q equal (x, y, z)
      // let point P equal (x1, y1, z1)
      // Solve this equation: n . (Q - P) = 0 , with the above values

         -- first just fill in the values
         (n1,n2,n3) . [(x,y,z) - (x1,y1,z1)] = 0

         -- simplify between the brackets by subtracting P from Q
         (n1,n2,n3) . [(x-x1), (y-y1), (z-z1)] = 0

         -- expand the dot product
         n1*(x-x1) + n2*(y-y1) + n3*(z-z1) = 0

         -- simplify by distributing the n's
         (n1*x - n1*x1) + (n2*y - n2*y1) + (n3*z - n3*z1) = 0

         -- since the only variables in there that we do not know the
            values of are Q(x,y,z) we move everything we do know to
            the right side of the equation.  We do this by adding all
            the expressions like (n1*x1) and (n2*y1) and so on.

         n1*x + n2*y + n3*z = n1*x1 + n2*y1 + n3*z1

         -- Now the left side of the equation look familiar right?
            We get the components of the normal vector put the
            coefficients x, y, and z in front of them.  And then
            the right side of the equation is basically the dot
            product of the normal vector and point P.

The cross product and dot product go hand in hand when integrating hidden surface removal and lighting in a 3D engine. And believe it or not I think that is all I have to talk about for vectors. There is so much more but this tutorial has gotten much longer than I planned.

Good luck.

- Brandon Williams