As a member of the LaTeX3 Project, my advice would be to totally ignore LaTeX3 when looking at books on LaTeX at the moment. There are two reasons. First, none of the books I know of cover anything about the new work of the LaTeX3 Project. Secondly, and really much more importantly, the 'user level' experience of LaTeX is not likely to change substantially even if/when LaTeX3 is released. So what you can learn now about LaTeX2e should in the main remain relevant for a long time.
So which book to buy? If it's just one I'd go for The Guide to LaTeX: this is intended as a guide for new users, and balances nicely the need to cover the basics with enough additional material to get going. On the other hand, I tend to think of The LaTeX Companion as more of a reference work. It also suffers a little from the changes in packages over time: there are a number of significant new LaTeX2e packages that are not in The LaTeX Companion.
TeX by Topic and The TeXBook are very good to understand TeX, but as starters I wouldn't recommend them for new LaTeX users. You can read TeX by Topic electronically, and as it is more of a programmers reference I find that quite convenient: I tend to want to look up a single TeX primitive at a time. If you are going to buy one in print, I'd probably go for The TeXBook as it is more narrative than TeX by Topic. The TeXBook is alos not intended purely for programmers (at least in theory!), while I'd say TeX by Topic is really not a users book.
(I guess I could be called a member of one of the teams ;-) this is my view)
I thought of staying out of this debate, but perhaps some words of clarification or, let's say, some thoughts are in order after all.
LaTeX3 versus pure Lua
First of all this is the wrong question imho: LaTeX3 has different goals to LuaTeX and those goals may well be still a defunct pipe dream, but if so they are unlikely to be resolved by pure Lua either.
So if one wants to develop an argument along those lines then it should be more like "Why does LaTeX3 use an underlying programming language based on eTeX and not on LuaTeX where a lot of functionality would be available in a "simpler" way?"
But LaTeX3 is really about three to four different levels
- underlying engine
- programming level
- typesetting element layer
- designer interface foundation layer
- document representation layer
See for example my talk at TUG 2011: http://www.latex-project.org/papers/
Here is a sketch of the architecture structure:
The chosen underlying engine (at the moment is any TeX engine with e-TeX extension). The programming level is what we call "expl3" and that is what I guess you are referring to if you say "LaTeX3" (and I sometimes do that too). However, it is only the bottom box in the above diagram. The more interesting parts are those that are above the programming level (and largely a pipe dream but moving nicely along now that the foundation on the programming level is stable). And for this part there is no comparison against Lua.
Why use LaTeX3 programming over Lua, when LuaTeX is available?
To build the upper layers it is extremely important to have a stable underlying base. As @egreg mentioned in chat: compare the package situation in 2.09 to the package situation in 2e. The moment there were standard protocols to build and interface packages the productivity increased a lot. However, the underlying programming level in 2e was and is still a mess which made a lot of things very complicated and often impossible to do in a reliable manner. Thus the need for a better underlying programming layer.
However, that programming layer is build on eTeX not because eTeX is the superior engine (it is compared to TeX but not with respect to other extensions, be it Lua or some other engine) but because it is a stable base available everywhere.
So eTeX + expl3 is a programming layer that the LaTeX3 team can rely on of not being further developed and changed (other than by us). It is also a base that is immediately available to everybody in the TeX world with the same level of functionality as all engines in use are implementing eTeX extensions.
Any larger level of modifications/improvements in the underlying engine is a hindrance to build the upper layers. True, some things may not work and some things may be more complicated to solve but the tasks we are looking at (well I am) the majority are very much independent of that layer anyway.
To make a few examples:
good algorithms for automatically doing complex page layout aren't there (as algorithms) so Lua will not help you here unless somebody comes up with such algorithms first.
something like "coffins", is thinking about approaching "design" and the importance here is how to think about it, not how to implement it (that comes second) -- see Is there no easier way to float objects and set margins? or LaTeX3 and pauper's coffins for examples
Having said that, the moment LuaTeX would be stable similar to eTeX (or a subset of LuaTeX at least) there might well good reasons for replacing the underlying engine and the program layer implementation. But it is not the focus (for now).
Why use LaTeX3 to program at all? Why not turn its development into the development of a LaTeX-style document design library for LuaTeX, written in Lua?
Could happen. But only if that "LuaTeX" would no longer be a moving target (because LaTeX3 on top would be moving target enough).
Side remark: @PatrickGundlach in his answer speculated that this
answer that the LaTeX3 goal is backwards compatibility. Wrong. The
same people that are coming down on you very strong about
compatibility for LaTeX2e have a different mindset here. We do not
believe that the interesting open questions that couldn't get resolved
for 2e could be resolved in any form or shape with LaTeX3 in a
document-compatible manner.
Input compatible: probably. But output compatibility for old
documents, no chance if you want to get anything right.
But in any case, this is not an argument for or against implementing the ideas we are working on one day with a LuaTeX engine.
Is the separation of LuaTeX and LaTeX3 a result (or at least an artifact) of the non-communication among developers that Ahmed Musa described in his comment to this answer? What kind of cooperation is there between these two projects to reduce duplication of effort?
As I tried to explain, there is not much overlap in the first place. There is much more overlap in conceptual ideas on the level ConTeXt viz. LaTeX.
An even more fantastical notion is to implement every primitive, except \directlua itself, in terms of Lua and various internal typesetting parameters, thus completely divorcing the programming side of TeX from the typesetting side.
That brings us to a completely different level of discussion, namely is based on LuaTeX, or anything else for that matter, a completely different approach to a typesetting engine possible? That is a very interesting thought, but as @Patrick explained it isn't done with leaving TeX to do the typesetting and do everything else in a different language. So far such concepts have failed whether it was NTS or anything else because fundamentally (in my believe) we haven't yet grasped how to come up with a successful and different model for the TeX macro approach (as ugly as it might look in places).
Best Answer
I'm going to expand on my comment and turn it into an answer.
I take issue with the word abandon in your question. Using
xelatexandlatex3does not mean that you have to abandonpdflatexandlatex2e. It is entirely possible to use them alongside each other. The differences at the document level are such that it is relatively straightforward to remember "This is apdflatexdocument" and "This is axelatexdocument". I do my lectures withxelatexbecause I want to be able to useunicode-math. I tend to write my articles withpdflatex- see below - though for the fun of it I tried an article withxelatexand using unicode symbols and really quite liked the readability of the source code.So when there's functionality that I would like to make use of, then I'm happy to use
xelatexorlualatexorlatex3. If I don't need them, then at the moment I'll probably not use them so that I keep my options open. As I see more "here's what you can do" on this site, then I find myself more and more using them to make use of what I learn here. Some is a bit silly (such as my use ofunicode-math) but others are more serious.The one thing that I really don't take any notice of is the compatibility with journals or the arxiv. This is for two reasons:
If a journal actually accepts my article then (within reason) I'm happy to modify it to suit their bizarre style requirements (and before you ask, all journals have bizarre requirements).
Before a journal accepts my article then I'll not know what bizarre requirements I'm going to have to meet so I'd rather make my life easier here and now than try to guess what I might have to do when it is accepted.
The point of that second one is that I spend a lot of time writing an article. I'm often doing the maths as I actually write it because it is often only when I see it written down that I see whether or not my argument worked - when it's in my head then I gloss over all the technicalities. So it's not "prove result then write article" it is "prove result while writing article". This means that the writing stage is longer and more involved than if I worked the other way and so it is all the more important that the writing be easy and not get in the way. So being able to make use of great tools and stuff is more important than not annoying some future copy-editor.
So if you see an example of something on this site and you think "That would be just perfect for what I'm trying to do", don't let the fact that it uses
xelatexorlualatexorlatex3put you off. Just think of it as another LaTeX package that you use for some of your documents, but not all of them. It's no big deal.