The concept of infinity is deeply ingrained in mathematics. It is hard to see how mathematics could exist without such a concept, as such a concept as basic as counting is based on the implicit assumption that each number has a successor (see the Peano postulates for more information).
Infinity has been a part of mathematics since the ancient Greeks. For example, Aristotle criticized Pythagoras and the Pythagoreans for believing that infinity is a "self-subsistent substance" instead of an attribute of something else, which is what Aristotle believed. He discussed infinite time, infinite magnitude, and infinite divisibility, but felt that infinite magnitude was an illusion. Many of the Greeks had problems with the notion of infinity; Zeno's Paradox illustrates such difficulties well.
Throughout this site, there are references to infinity on many otherwise unrelated pages; for example, the Fibonacci numbers page, and the natural numbers page. It is difficult to find a branch of mathematics in which there are no references to infinity; for example, the set of natural numbers is infinite.
Infinity's properties are very different from those that we are used to in the finite world of numbers. You might want to start by reading the Hilbert's hotel page. Having done that, you might think "Wow, this is pretty easy: ∞ + 1 = ∞, ∞ × 2 = ∞, ∞ + ∞ = ∞, even ∞ × ∞ = ∞! Why don't they teach these mathematics facts in grade 1?"
While the behaviour of ∞ is very different from what we usually see in finite mathematics (in finite mathematics, we always get a large number when we add 1 to it, but infinite arithmetic is different), the "facts" you discovered above may not be strictly true in all situations. We'll need to take a deeper look into what infinity means.
The simplest example of an infinite set is the set of the natural numbers, N. No matter how high you count, you can always count higher; that is, you can always find the successor of any natural number. One might ask whether there is more than one kind of infinity. This is the basis of Galileo's Paradox. Galileo's paradox states that there are more natural numbers than there are square numbers, because not all natural numbers are square, but since both sets are infinite, both sets have the same number of members. You may also wonder if there are "more" real numbers than natural numbers, even though both sets are infinite. In order to solve these problems, we need to take a deeper look at what we mean by infinity.