The obvious answer is hydrogen and helium plasma but the nuclear fusion can also create heavier elements. Are these heavier elements a significant portion of the core?
As said in dmckee's answer, no, the core of the Sun is much too cool (about ~15 000 000 K) to burn any other than hydrogen into helium. The triple-alpha process, which converts helium into carbon, only kicks in somewhere around 80 000 000 K, depending on density.
That said, the CNO cycle does modify the internal abundances very slightly. That is, the CNO cycle is a catalytic process, so the abundances of those elements are driven to the values at which the reaction proceeds at an equilibrium rate. Glancing at a solar model I have lying around, this leads to about a 10% enhancement of the central nitrogen abundance, and a corresponding decrease of the carbon and nitrogen abundances.
Do the heavier elements "sink" to the "bottom" of the core, like iron has during planetary formation?
Actually, yes, they do! We refer to these processes as atomic diffusion. The one you're thinking of is known as gravitational settling. In short, yes, heavier elements "sink" towards the centre. This process takes a long time to make a meaningful difference: billions of years. For the metals, it isn't important, but it is actually important for the helium abundances. There was a minor revolution in the mid-1990's when this effect was included for the first time, and it led to a much better fit of solar models with respect to helioseismic observations.
Presumably, during the Sun's formation it would have accreted heavy elements made by previous generations of stars - does this just get added to the mix?
You're quite right: the Sun's initial composition reflected that of the nebula from which it was born, itself a product of whatever star(s) preceded it. The primordial mixture is expected to be fully mixed before a star starts burning hydrogen into helium. The reason is that the star (or, at least, our models) goes through a phase where the whole star is convective, so everything gets churned up and homogenized.
Best Answer
No, the Sun is not thought to have formed around a solid core, and solids would not exist at the temperatures and pressures at the centre of the protosun. The Sun formed simply from the gravitational collapse of a large cloud of gas.
The situation for Jupiter is different because far out in the circumstellar disc of the forming solar system, it was cool enough for the condensation of solids. The core accretion model is where gas giant planets begin their lives through the growth of a small (well maybe 10 Earth masses!) rocky/icy core, this is later followed by a brief and rapid accretion of the gaseous envelope.
It is currently not known whether Jupiter has a rocky core or not. That is one of the key questions that it is hoped the Juno mission will answer.
Recent years have seen the re-emergence of the thermal instability model for the rapid formation of gas giants. Such giants would not have a solid core and this formation mechanism is more akin to the way that the Sun formed.
As to whether there could be observational evidence that the Sun did not form around a solid core, I am doubtful. A solid core would of course vapourise at the high temperatures inside the Sun, but its enhanced mean atomic mass could remain, altering the structure of the core, nuclear reaction rates etc. The trouble is, that the vaporisation would occur whilst the contracting Sun was still fully convective, effectively mixing everything up throughout the Sun before nuclear reactions began.