Some may find this rather strange, as the demand side of the equation is very predictable. Surely, governments ought to be able to plan for such predictable demand? Consider what my friend MJ recently pointed out about food :
1. The total demand for food rises extremely slowly YoY. No matter how hard I try...bouts of fasts and feasts...the standard deviation of my daily intake of total intake of food would be small. Higher intake of one item would be compensated by lower intake of something else. It gets even smaller over a population.
2. The shift in preference for one food over another in a society happens at glacial speed (barring temporary shortages and bounties...also passing fads). Take the meat preferences in US as an example: i) Despite all the 20th century prosperity in US and the mechanized farming, pork remained the most preferred meat and only near 1950 did beef surpass it in quantity consumed. ii) By 70's, the issue of health (lifestyle diseases) and white meat over red meat had become widely understood. However, that made a very small dent (2%) on relative beef and chicken (of Fish, if you like) consumption by 90's (I don't have comparable data for this decade).
Considering these 2 facts, changes in medium term (3-7 years) demand maybe better explained by specific changes rather than a macro trend.... A Pepsi or Haldiram initiative is more potent than the long-term social change.
My view is that volatile commodity prices can be explained due to the effects of supply shocks. Probably the two most important measures for predicting how volatile a commodity can be are the "spare capacity" and the "stocks-to-flow ratio".
Imagine that wheat has a peak capacity of around 100M kg and a demand of 90M kg . If the "flow" increases (glacially - over a decade) to 95M kg, the spare capacity halves. Due to the way probability and statistics work, the frequency of shortages will increase exponentially(much more than double) resulting in much more volatile prices due to supply shocks.
Existing stock can stabilize prices in the face of supply disruptions. However for food stocks can be low if the cost of stockpiling is high or if it is perishable. Thus, even under a glacial rate of change in demand and unchanged variability of supply, commodities with low spare capacity can tip into severe volatility due to seemingly benign shortages.
If you add high liquidity and easy credit to this scenario, jumpy traders and other economic actors make things worse in the futures market through speculation or in real markets by hoarding.
Interestingly, the stock-to-flow ratio is also a great measure of what makes a good pricing commodity. A commodity with a high stock to flow ratio is useful for pricing because its value is stable and not volatile.
Gold has by far the highest stock-to-flow ratio because very little of it is mined and it is never really "consumed". Thus its flow is tiny. Practically all the gold mined is still in existence. Its desirability underpins its demand. This results in a stable prices when compared to other commodities. It is durable, divisible and has high value per unit weight. Thus, it makes for an excellent store of value compared to other commodities.
A credit system built upon gold has stable interest rates and efficient capital flows. Gold can be transported at relatively modest cost in response to global arbitrage opportunities in interest rates. This is why the British were able to run a huge global trading empire under the Gold Standard using a relatively small amount of Gold as base money in the Bank of England.
Silver is the next best pricing commodity. Although its stocks to flow ratio is much lower than gold, it is still quite high relative to other commodities. Its shares Gold's properties of durability and divisibility. However its relatively lower value per unit weight made it less useful for international capital flows and more useful for local exchange in small denominations, and for coinage. Thus silver was often called a common man's currency. The latter use declined with the general acceptance of self-liquidating Gold-backed Real Bills in industrial societies which replaced silver as the primary currency, even for local exchange.
Oil and other forms of energy have a very low stock relative to their enormous flows. This makes them much more volatile, which is why they are unsuitable as a pricing commodity.
Agricultural products are usually perishable and bulky and make woeful pricing commodities.
Other assets have some good properties (e.g. diamonds, livestock) but not others (not divisible, not durable, etc).