descriptive statisticspredictorterminology
I'm confused with this term: covariate. What is it? Is is just the observed outcomes of some random variables that contain information that could help us enhance our prediction of another random variable that we haven't observed yet? Why is it named so?
Also there seems to be another: independent variable. Independent of what? Why is it named so?
Some idea. Hopefully you'll get a few responses and all together there will be something useful.
(1) I usually use the term "main effects" to refer to a multivariate regression model before I've addressed interactions, or that part of a multivariate model that includes the variables but not the interactions. Thus in y ~ x + z + x*z, x and z are the main effects.
(2) It sounds like you're doing bivariate analysis to look for confounders that should be included in a final multivariate regression model. This isn't the best way to build a regression model. I'm not sure there is a best way, and it does depend on what you're trying to achieve with your model. If you're just trying to find the best estimate of your main variable of interest then include all/most variables, if you interested in a group of factors that are all together associated with the outcome some model building is needed.
(3) Some covariates turn out to be significant AND the X variable is also significant = the covariate is associated with the outcome, but this does not say anything about whether the covariate is associated with x. If it is associated with outcome and x it could be a confounder
(4) But what is the case if the some covariates are not significant, but the x variable is significant and even more significant than in the analysis without covariates? = is this the classic "suppressor variable" ? (http://www.uvm.edu/~dhowell/gradstat/psych341/lectures/MultipleRegression/multreg3.html) Here I'm assuming you mean statistically significant.
The word "sample" causes at least two different instances of confusion.
A (what the OP asks about)
The tag "Sample" here on CV starts by "A sample is a subset of a population": all possible elements included in any possible subset of a population can only be an event that is possible: hence the set of all possible events, can be called the "Sample Space" (the "Population Subsets Space"), because it is from that Space that the elements of any population subset can come.
Where does that leave us regarding the relation with the concept "outcomes"?
The population and its subsets do not consist of the numerical values that the elements of these subsets may take: these numerical values are assigned by the random variable that we have defined according to our needs.
To consider the trivial example, a series of coin-flips can be thought as a population of heads and tails. We define a real-valued random variable by, say, linking "Heads" with the number $5$ and "Tails" with the number "$17$". So the Sample Space will be "{Heads, Tails}", which will be the domain of the random variable, while the "outcome space", its range, will be $\{5,17\}$.
In other words, it is not necessary that "the function maps values to values" as the OP states. It can map anything to values.
And strictly speaking, a "sample" of, say size $3$ will be a set like "{Heads, Heads, Tails}", and not the set $\{5,5,17\}$. This latter set is produced by a specific random variable. Obviously, we could use another random variable and obtain a different numerical representation for the same sample.
In all, the Sample Space can be non-numerical while the "set of outcomes" of a real-valued random variable should be real-valued. To each realized sample from a population we can map infinitely many numerical sets. It is by no accident that the latter are properly called "a sample of realizations of a random variable", and not just "a sample from a population".
Assume now that we have a coin where on the one side it reads "$1$" while on the other it reads "$2$". So the Sample Space here has a numerical nature. Still we can define a random variable by mapping $1$ to $5$ and $2$ to $17$. Here too, the Sample Space $\{1,2\}$ will be different than the "Outcome Space" $\{5,17\}$.
Our sample of size $3$ (understood as a subset of the population) will here be the set $\{1,1,2\}$, while the "sample of realizations of the (specific) random variable" will be $\{5,5,17\}$.
B: Sample and Observation
In fields like medicine or biology, when we say "let's take a sample of blood", we mean "let's take blood once". If we wanted to put this in general statistical terminology, we would have one observation... because in general statistical terminology a "sample" is a set containing usually more than one observation (although it can contain only one).
So when somebody from these fields will say "I have available $n$ samples" - he just might mean, in general terminology, "I have available $n$ observations" or "I have available one sample of $n$ observations" -but someone else that is used in the more standard terminology, by the expression "I have available $n$ samples", she will understand "I have available $n$ sets each containing $m$ observations" -and usually $m\geq 1$. One can find this sort of confused communication in various posts here on CV.
ADDENDUM
Responding to the OP's edit in the question:
"Why not sample real numbers right away"? Because the world is not made by numbers. Actual data collection that describes the world is in many cases of qualitative nature. So, "separating samples and outcomes" follows the nature of things. Moreover, the act of mapping them to numerical values is a separate step, and as I have already mentioned, it is not a unique mapping. So it requires decisions to be made. And whenever decisions are involved, they better be clear and transparent so that they can be judged, assessed, and criticized. These "decisions" are, to begin with, the choice of the random variable we will use.
"Heads and Tails" exist irrespective of whether we want to study them. The "random variable" is a mathematical concept/tool which we project onto the real-world data in order to analyze and study them. So, samples, they exist. Random variables, they transform samples into something that we can handle using quantitative methods.
As to whether "samples are deterministic", nobody has ever decisively argued of whether there exists anything inherently stochastic in nature, or whether all our stochastic approaches are just a reflection of our ignorance, and/or of the limits of our measuring devices.
Best Answer
From Wikipedia:
Answering (some of) your questions:
Let's take a concrete example. Suppose you wish to predict the price of a house in a neighborhood, $\textbf{y}$ using the following "co-variates", $\textbf{X}$:
For a linear regression problem, $\textbf{y} = f(\textbf{X})$ the price of the house is dependent on all co-variates, i.e. $\textbf{y}$ is dependent on $\textbf{X}$. Do any of the co-variates depend on the price of the house? In other words, is $\textbf{X}$ dependent on $\textbf{y}$? The answer is NO. Hence, $\textbf{X}$ is the independent variable and $\textbf{y}$ is the dependent variable. This encapsulates a cause and effect relationship. If the independent variable changes, its effect is seen on the dependent variable.
Now, are all the co-variates independent of each other? The answer is NO! A better answer is, well it depends!
The area of the house ($x_4$) is dependent on the width ($x_1$), breadth ($x_2$) and the number of floors ($x_3$), whereas, distances to downtown ($x_5$) and hospital ($x_6$) are independent of the area of the house ($x_4$).
Hope that helps!