A Survey of Mathematics: Elementary Concepts and Their Historical Development by Vivian Shaw Groza (Holt, Rinehart and Winston, 1968) is a very interesting book. It is a textbook for an introduction to mathematics presented through the historical background of the development of the concepts.
The book is broken down into four parts: The Prehistorical Period, The Ancient Oriental Period, The Greek Period, and The Hindu-Arabic-European Period. Each of these four parts consists of several chapters presenting mathematical concepts with reference to the mathematics of the period in question. Included in the book are chapters on counting, sets, numeral systems, arbitrary bases, early computations, logic, mathematical systems, elementary Euclidean geometry, Greek arithmetic, the development of arithmetic, algebra, probability, calculus, and non-Euclidean geometries. The material is presented in a coherent manner and the historical context keeps the non-mathematical reader's interest. Includes various exercises, the answers to many of which can be found at the back of the book.
One thing to note about this book is that it is a mathematics book for people who like history, not a book about the history of mathematics, as suggested by the preface, which indicates that the book is the result of developing lecture notes used for a one-semester course in mathematics for a general education curriculum. Two points relevant to this purpose are worth pointing out. First, the book is structured along the lines of mathematical concepts with more fundamental ones occurring first; this means that concepts such as set theory, which are quite fundamental but not explicitly investigated by the ancients, appear early in the book. Second, the history portion contains some inaccuracies which, while not major, history buffs may pick up on.