You're used to dealing with numbers in base 10 -- where every "place" you move over to the left represents a multiple of 10, so that you get 1, 10, 100, 1000, etc.
In base 10 there are 10 digits, 0-9, that can be in each "place", and when you go over 9 you add one to the next place and set the current place back to 0.
Binary is just base 2, with the same rules. However, being base 2, there are only 2 digits -- 0 and 1. When you go over 1 in each "place," you add one to the next place and set the current place back to 0, just like with base 10. It just "turns over places" a lot faster since there are only 2 digits :) So to count in binary, from one to ten, you get:
0, 1, 10, 11, 100, 101, 110, 111, 1000, 1001, 1010
Each "place" represents a power of 2 instead of a power of 10, so the place values (right to left) are 1's, 2's, 4's, 8's, 16's, and so on.
Binary is a useful way of representing numbers because the two digits can be represented in electronic circuits as "on" (for 1) and "off" (for 0). That lets computers use electrical signals to do math, and it's the basis of all computer use. Really, it's just another way of representing numbers in a different base, but they're the same numbers. You can have a "base" of anything -- bases of 16 and 8 (hexadecimal and octal) are especially useful when dealing with computers also, since they're power of 2 bases that require less digits than binary. For example, the number 10 in different bases looks like:
Base 10: 10
Base 2: 1010
Base 8: 12
Base 16: 0A (since we only have 0-9 as numbers in our writing system, we use A-F to represent 10-15 in hexadecimal!).
They're all 10, just different ways of writing them.
Hope that helped...