Interesting equations. The first says that the points l,o,q,c are coplanar, but I'm not sure what the second means.
You can solve a system like this using fsolve. Since fsolve requires a function f(x) with vector x, you first need to combine your unknowns in a vector, e.g., x = [q; c]. Note: I am assuming your vectors are all column vectors. Create a function
function y = myfun(x,l,o)
q = x(1:3); c = x(4:6);
y = [dot(cross(l-o,q-o),c-o)
abs(o-q)*dot(l-q,q-c) - abs(l-q)*dot(o-q,q-c)];
To solve your system of equations, you need to find an x that makes y equal to zero. Save this function in a file.
Then make an initial guess for your solution, e.g., q0 and c0. Finally, solve:
x0 = [q0; c0];
xsol = fsolve(f,x0);
Best Answer