MATLAB: Rand function is not uniform in large intervals

MATLABrand function

I am using rand function to generate uniformly distributed random numbers in the interval [10e-6 and 1.] But the function generates the nos. which are close to 1 (RATHER THAN BEING UNIFORM IN THE ENTIRE INTERVAL]. I have tried with 10 nos. and 100 nos. But I found that most the nos. generated are close to 1. Then, how it will be a uniform distribution??

Best Answer

Sigh. I think this is a misunderstanding of what uniformly random means. A fairly common one too. Since you seem not interested in showing your code, one can only guess the problem though. There can be no certain answer if I do not see your code. (addendum: The OP has since added a comment that indicates my conjecture is exactly on target.)

You are generating numbers uniformly random with a target interval [1e-6,1].

A uniform distribution implies that for ANY sub-interval of fixed width that is contained in the global window [1.e-6,1] (so assume a sub-interval [a,b]) where we have

1e-6 <= a <= b <=1

then the expected number of events we will observe should be:

(b - a)/(1 - 1e-6)

If you will generate N samples, then the expected number of events in the sub-interval is:

N*(b - a)/(1 - 1e-6)

So expect to see a number of events that are proportional to the sub-interval width. You won't seee exactly that many, since this is a random sampling.

So a uniform random sampling on the interval [0,1] would have roughly 10% of the samples in each bin [0,0.1], [0.1,0.2], [0.2,0.3], etc.

Now in your case, you are sampling on the interval [1e-6,1]. You find that very few samples occur right down at the bottom end, say between [1e-6,1e-5].

Lets use the rule above to see what fraction of the samples SHOULD occur in that interval. Lets say that we generate a sample size of 1000 values in the overall interval. Seems pretty big to me.

1000*(1e-5 - 1e-6)/(1 - 1e-6)
ans =
        0.009

Hmm. I only expect to see 0.009 samples in that sub-interval, whereas I would have expected to see

1000*(1 - 0.9)/(1 - 1e-6)
ans =
          100

So 100 events in the subinterval [0.9,1].

Is this truly uniform sampling? YES!!!!!!!!! Of course it is! You need to understand that the first interval I showed is a terribly tiny interval.

If you asked to generate a sampling that is uniformly probable over that region, but what you REALLY wanted was some sort of sampling that is uniform in a log space, then you needed to use a proper random sampling scheme!

For example, try this:

R = 10.^(rand(1,1000)*6 - 6);

Look at some percentiles of this sampling scheme:

     Min  1.005e-06
    1.0%  1.144e-06
    5.0%  2.036e-06
   10.0%  4.529e-06
   25.0%  4.117e-05
   50.0%   0.001265
   75.0%    0.03185
   90.0%     0.2554
   95.0%     0.5309
   99.0%     0.8607
     Max     0.9928

It is NOT uniform, at least not in the domain [1e-6,1]. But the log10 of those numbers WILL be uniformly distributed. So, we will expect roughly 50% of the log10 values to be less than -3.

     Min     -5.998
    1.0%     -5.941
    5.0%     -5.691
   10.0%     -5.344
   25.0%     -4.385
   50.0%     -2.898
   75.0%     -1.497
   90.0%    -0.5928
   95.0%     -0.275
   99.0%   -0.06513
     Max  -0.003133

Again, it won't be perfect. But a sample size of 1000 is not really that huge. These predictions only become valid in the limit as N grows to a really large number.

Again, it is just a wild guess.

Related Solutions

MATLAB: Generation of random numbers

I think what you're trying to do is done using randn, so that

dd=(10^(4) - 10^(-4)).*randn(1000,1) + 10^(-4)
hist(dd,1000) # To see it in a histogram with 1000 bins

Note the range of x-axis

MATLAB: Generating dispersed (non-integer) random matrix/array that sums to a particular value

I think you do not understand what you are asking.

randfixedssum indeed produces results that are uniformly sistributed within the sub-set in question. That is, any point in a 10 dimensional space that satisfies the requirements of a fixed sum is equally likely to arise.

However, that does not mean that it is at all probable you would find something that satisfies your goal, of "dispersion".

For example, suppose you were to choose one element that is greater than 0.5? Then the probability that the other 9 elements were ALL small enough that the sum is 1, is pretty low. In the 9 dimensional space that remains, that event would be actually very uncommon.

Thus, you want to generate 10 numbers, all of which lie between 0.05 and 0.9, such that the sum is 1.

Suppose, just suppose that one of the numbers was say, 0.6? Now what are the odds that you can find 9 other numbers that make the total sum exactly 1, but none of them are less than 0.05? SURPRISE! It can never be done.

In fact, if any simgle element was any larger than 0.55 in this example, your goal will never be doable. So if one element is as large as even 0.55+eps, it is mathematically impossible to find 9 numbers, all of which are between 0.05 and 0.9, such that the sum is 0.45-eps.

Next, suppose one element was even as large as 0.5? Just one element that large?

Now the other 9 elements must all be very close to 0.05. What is the probability of that event? Not surprisingly, it is pretty darn small. I can compute the actual probability of such an event to happen if you need. Being too lazy to think at this time of day...

X = randfixedsum(10,10000000,1,0,0.9);
sum(max(X) >= 0.5)
ans =
      195844

So 1.96e5 such events in 1e7. A little under 2% of the time. As expected, a rare event, and that is EXACTLY as it should be.

You ask for dispersion. But you don't seem to understand what dispersion means or what it implies in this context.

If I look at the distribution of the maximum of all 10 elements, I get something that is actually pretty reasonable.

X = randfixedsum(10,10000,1,0.05,0.9);
   Min     0.1207
  1.0%     0.1342
  5.0%     0.1445
 10.0%     0.1524
 25.0%     0.1674
 50.0%     0.1884
 75.0%     0.2167
 90.0%     0.2503
 95.0%     0.2738
 99.0%     0.3143
   Max     0.4039

Most of the time, we get a maximum value that is pretty small in context. And that is because the sample truly is uniformly distributed around the constraint space. One point in that space is equally as likely to arise as any other point. But that does NOT mean that the maximum is ever likely to be larger than 0.55. In fact, that would be an impossible event.

Suppose instead, that we change the way things were generated? Now, instead of requiring that the min be 0.05. Just make it 0. How do the statistics change?

X = randfixedsum(10,10000,1,0,0.9);
   Min     0.1395
  1.0%     0.1681
  5.0%     0.1902
 10.0%      0.205
 25.0%     0.2353
 50.0%     0.2784
 75.0%     0.3359
 90.0%      0.401
 95.0%     0.4492
 99.0%     0.5479
   Max     0.8123

As you now see, the maximum element is now considerably larger. In the same size sample, I once got something as large as 0.8123. There is now much more room for those "dispersed" events to arise.

Best Answer

Related Solutions

MATLAB: Generation of random numbers

MATLAB: Generating dispersed (non-integer) random matrix/array that sums to a particular value

Related Question