It seems that you are looking for mathematics that is not too technical. In this regard, I do recommend the textbook 'Comprehensive Mathematics for Computer Scientists' because it contains the type of theory that forms the foundations of mathematics - i.e. sets and logic - without going into numerical analysis. Logic and set theory can be intuitive and fun and is definitely useful for application in other subjects such as computer science and philosophy.
On the other hand, theoretical mathematics that is abstracted from numbers, such as linear algebra (chapters 20 -25), can be difficult to master if you are not dedicated, as there are many complicated concepts to understand. For a student with no math foundation, many of the chapters in this textbook will be fast-paced.
If you are looking to "reintroduce basic maths", then this book is not suitable. A better textbook would be one that is aimed at high-school students or a first-year introductory course. The level of math that you will be at once you have mastered the book will be about the end of a second-year pure maths course at university. You can expect to require 5 hours per week for a year $\approx$ 250 hours in total.
An alternate solution, which is a "self-contained" book and requiring "minimal to no previous mathematical background" is to find a descriptive coffee-table style book on mathematics, such as 50 mathematical ideas you really need to know by Tony Crilly (London: Quercus). In my opinion, the more description that the book gives and the fewer formulae, the easier it is to understand what is going on. This book can easily be read by dipping into and out of different chapters, which each take about half an hour to read and understand (there are 50 chapters).
I'm kind of in the same 'boat' as you; I've graduated but continue to study in my own time as a leisure activity. I completely feel the same way in that I now self-study but find it difficult to seek critique of my newly acquired knowledge.
These are suggestions and probably will not be a full answer to your question:
1) Participating in this community,
2) Seeking people in the same position as you,
3) Intense 'Googling' and wikipedia,
4) University notes and solutions,
5) 'Crack on' and Critique yourself.
1) As @omnomnomnom says, you are welcome to use this community to verify and give suggestions to your answers by using the relevant tags. You will find that some people have used the materials that you are referring to and will be able to give you hints/solutions/intuitions/motivations etc.
2) You will find that there are many people who are in the same position as you. And, that there will be a similar number of people who seek support and guidance with their study. Have a look if there are similarly minded people in your community (or even f***book) that you can buddy up with.
3) In my experience, it's easier to look for a list of books, in the field of chosen study, and find whether there is solutions for any of the books on that list. Then you can optimize for the best book against the most resourceful solution-base and usually find almost exactly what you want.
Or even work the other way; You could search the internet for solutions in a chosen-field and then find the book the solutions are for. (I did exactly this for Guillemin's & Pollack's Differential Topology)
A tip for finding books is using wiki; search for a subject that you think you might be interested in on wiki, then look at the references and then check these references with review forums etc.
4) As you said there are plenty of sources for mathematical study but try to use university websites. If they are available to the public then it's fair game. They will often be accompanied by tutorials, solutions and amendments.
5) You'll get to a point where you'll be able to see whether your arguments and solutions are correct, overkill, not quite there or just plain wrong. It takes time. Look for counter-examples, always ask what if (and what if not) and make sure a logical statement means what you think it means. Most importantly, you should try to understand (properly) every new 'thing' before you move onto the next.
Lastly, ask your own questions. And, try to answer them. This way you (sort of) have a feeling of whether your answer is rigorous enough. And always write these down, you may come back to them and want to improve (or re-write) them.
Disclaimer: I offer suggestions based on my experience.
Best Answer
Before you take on the difficult task of undertaking a lengthy study of mathematics, you'll need to ask yourself why you are doing it. For instance, if you are interested in a career in the sciences, then the mathematics you will need to learn will be far more advanced than if you went into (for example) business management.
If indeed you are interested in studying mathematics beyond a year-11 level, then you'll need to direct your studies as necessary. Beginning with basic algebra and arithmetic (for example, learning what functions are, solving quadratic equations, doing basic statistics), you will need to move on to calculus, which is often introduced at the year-11 level.
More precisely, you will need to begin studying algebra and trigonometry in good detail. I will recommend the textbooks written by James Stewart, as I find them extremely comprehensive, and also quite challenging, which I believe is a great trait in a textbook for a student who is willing to learn mathematics. Stewart's Algebra & Trigonometry is great for an introduction to trigonometry and algebra, and for a more advanced approach, try Stewart's Pre-Calculus, which contains everything you need to learn before you begin a study in calculus.
If, perchance, you master these techniques, many of which you may already be familiar, and are confident enough to begin a course in calculus, I direct you to this post.