Friday, December 14, 2012

Andrew Lo on Mathematical Models

If Adam Smith had a mind meld with Charles Darwin, Andrew Lo might result. A professor at MIT's Sloan School of Management, Lo is known for his multidisciplinary approach to finance, using everything from statistical analysis to neuroscience to better understand the markets. One of his most important ideas involves the "adaptive markets" theory.For a long time, many economists believed in the "rational markets" theory, which posited that all available information was reflected in a stock's price and investors were rational and so, therefore, were prices. Lo  believes markets are less like rule-based physics and more like messy biological systems. Market participants aren't coldly rational creatures but squirmy, evolving species interacting with one another in a primordial sludge of money.

"Blaming quantitative analysis is akin to blaming E=MC2 for nuclear meltdowns," says MIT Sloan School of Management's Andrew W. Lo.

Instead of blaming quantitative analysis he says, a more productive line of inquiry might be to look deeper into the underlying causes of financial crisis, which ultimately leads to the conclusion that bubbles, crashes, and market dislocation are unavoidable consequences of hardwired human behavior coupled with free enterprise and modern capitalism. However, even though crises cannot be legislated away, there are many ways to reduce their disruptive effects. Lo will conclude his talk with a set of proposals for regulatory reform.

By tracking the data trails left by this Darwinian process, we might be able to get a better picture of how markets really work. The U.S. Treasury buys it; Lo helped set up the new Office of Financial Research, which aims to provide better data and insights about the industry. "Policymakers are always looking for the financial-system equivalent of the MRI," said Treasury Secretary Tim Geithner at the launch of the OFR last year. Digging in the financial dirt may be the way to get it.

Andrew Lo, a financial economist and a leading authority on Hedge Funds.Lo is also the director of the MIT Sloan School of Management’s Laboratory for Financial Engineering

In 2009 Andrew Lo delivered the I.E. Block Community Lecture.Andrew Lo, the Harris & Harris Group Professor in the Sloan School of Management at MIT, titled his lecture "Kill All the Quants? Models and Mania in the Current Financial Crisis."Lo, who is also director of the MIT Laboratory for Financial Engineering, began with a brief history of the crisis, the seeds of which were sown during the period of low interest rates in the late 1990s. The ensuing housing boom was sustained by a proliferation of adjustable-rate mortgages and subprime mortgages (a.k.a. "liar loans") extended to borrowers lacking the means to repay, as well as by the wholesale issuance of mortgage-backed securities (MBSs) by financial institutions, including the government-sponsored enterprises known as Fannie Mae and Freddie Mac. Such instruments could be leveraged investors could use them as collateral against which to borrow more money to make more such investments, which could then be reused in the same fashion, and so on. Their acceptability as collateral was due in large part to the AAA ratings slapped on them by well-established credit-rating agencies, such as S&P, Moody's, and Fitch, and to the availability of default insurance in the form of credit-default swaps. But the wealth so created could not survive the interest rate increases of 2004 and the consequent decline of housing prices from 2006 highs. Default rates increased, inflicting losses on investors, dealers, insurers, and mortgage originators, which led to further declines in home prices, and so on. The resulting "death spiral" continues.

When it came to assigning blame, Lo named the usual suspects. Homebuyers deserve a share for assuming loans they couldn't hope to repay, as do the real estate agencies and mortgage companies that rewarded their brokers for persuading such buyers to assume unserviceable loans. The investment banks that purchased their contracts, bundled them into MBSs, and persuaded the rating agencies to label them AAA-the better to sell them to the proverbial "widows and orphans" funds are surely at fault as well, as are the investors who failed to peer behind the ratings. Also blameworthy are the insurance companies that guaranteed payment; the rating agencies that gave apparently inflated ratings; the regulators (the SEC, the Fed) that looked the other way; the politicians and government-sponsored enterprises that promoted the "ownership society"; and the business media that (mostly) applauded the orgy of "wealth creation."

All this was familiar ground. Lo was really more interested in assessing the blame to be borne by risk managers and other quants. To that end, he had to explain how some of those quants had contrived to turn mortgages with unappealing risk/return ratios into MBSs that sold like hotcakes to investors of every description. That was an achievement worthy of Jimmy (The Greek) Snyder, who invented the point-spread wager as a way of enticing the betting public to gamble on football and basketball games in which---as is usually the case one of the teams is a clear favorite to win. The packagers reduced the risk in part by spreading it geographically: On the basis of evidence that epidemics of homebuyer defaults tend to be regional, they assembled mortgages from every time zone into MBSs that were or seemed to be inherently less risky than the individual mortgages. Mainly, however, they did it by separating the underlying mortgages into "tranches" with different risk/return characteristics.

Andrew Lo was included in the Times Magazine's "World's 100 Most Influential People" list.

Lo explained the mechanics of the tranche system with an extremely simple model. The ingredients were a pair of identical $1000 IOUs, to be redeemed on a certain date in the not too distant future, each with a 10% chance of default. The expected value of each individual asset, which he equated with its price, is 90% × $1000 + 10% × $0 = $900. Under the assumption that defaults are statistically independent events, a portfolio containing both assets should be worth 81% × $2000 + 18% × $1000 + 1% × $0 = $1800, because either, neither, or both borrowers might ultimately default.But there are other ways to package the same pair of assets. Modern financial institutions, Lo said, are more likely to create two dissimilar asset-backed securities (ABSs), one outranking the other. In his example, purchasers of the "senior-tranche" ABS stand to collect $1000 unless both borrowers default, while "junior-tranche" purchasers collect $1000 only if both borrowers repay. The expected value of the senior-tranche ABS is thus 99% × $1000 + 1% × $0 = $990, that of the junior-tranche ABS 81% × $1000 + 1% × $0 = $810.

The senior-tranche ABS is thus a low-risk/low-return investment, suitable for purchase for the benefit of widows and orphans, while the junior-tranche ABS is a high-risk/high-return opportunity attractive to hedge funds and the like. The "high rollers" who invest in such funds understand the need to bear extraordinarily high risk in order to obtain extraordinarily high returns on their investments, and they actively seek out opportunities to do exactly that. But what happens if the defaults cease to be statistically independent? If the two become perfectly correlated, either paying or defaulting together, they will have the same expected value: 90% × $1000 + 10% × $0 = $900. The senior-tranche investors have then paid $990 for an asset worth $900, while the investors in junior-tranche (and riskier) ABSs find themselves in possession of assets worth $900 for which they paid only $810!
Simple though his model was, Lo confirmed during the question-and-answer session that the illustrated effect is entirely real: Hedge funds that invested heavily in the high-risk tranches have (on balance) been hurt less than more risk-averse investors. That may seem unfair, but it is not inconsistent with the history of financial crises. Fortune smiles on those who buy low! Despite the large number of hedge funds shut down during the crisis, Lo said at the reception following his talk, he expects that new ones will form in coming months, taking advantage of the many high-risk/high-return opportunities so reliably spawned by disorderly markets.

Lo found no fault with the bundling of loans to create a more attractive line of financial products. He expressed concern, rather, with those in the industry and here he specifically included quants who were in a position to see, yet failed to see, the danger with which the industry was so obviously flirting. But he did not hasten to judge even then, on the ground that it isn't always easy to see the obvious while striving to observe something else. He emphasized this point by showing a well-known video (which about a tenth of the audience had seen previously) in which two teams of three students each one dressed in white, the other in black strove to pass a pair of basketballs back and forth without fumbling. The audience was to count the number of times the white team passed the ball successfully from one member to another, as Lo called out random numbers to confuse the counters.
When the action ceased, Lo asked for the white team's total, and for anything else the audience had seen. Many laughed when a voice from the back of the room asked whether he meant "the guy in the gorilla suit." For a surprising number of those present, intent as they were on counting passes, had failed to see anything unexpected. When Lo reran the video, however, it was impossible to miss the person in a gorilla suit walking brazenly out into the middle of the action, pausing to beat his chest several times before exiting stage right. It was a memorable way to make the point that it's easy to see what you expect to see, and surprisingly difficult to see the unexpected. Because people in the finance industry hear no end of talk about the heroic quantities of wealth they are creating, and little if anything about the danger they pose to the economy as a whole, it is hardly surprising that the latter should be all but invisible to them.

Lo quoted with approval Charles Perrow, a systems-behavior expert who, after studying the tragedies at Chernobyl and Bhopal, as well as the near tragedy at Three Mile Island, formulated a theory of "normal accidents" in high-tech industries. The gist is that catastrophic failures are to be expected in such industries: Complex nonlinear systems (which are famously hard to predict) abound, and the various subsystems are so tightly coupled that a failure in one is likely to produce failure in others. And, because they cannot be prevented, they should be planned for! To this, Lo would add that human failure is the main cause of such breakdowns, and that the absence of negative feedback (of the sort that results in the identification and correction of mistakes) over an extended time is a virtual guarantee of human failure. Finally, he said, all of these observations apply directly to the finance industry, in which investors, managers, legislators, and regulators receive little constructive feedback.

It's not a tragedy when hedge funds lose money, Andrew Lo pointed out in Denver; it is a crisis when banks or retirement funds lose money. The key to avoiding future crises? More training in financial mathematics and engineering, not less.

Wondering whether the crisis could have been avoided, Lo described the probable fate of various operatives, had they known with certitude in 2005 that a crisis was on the way. A Wall Street CEO might have shut down one or two groups in the firm that were earning money hand over fist, as were similar groups at rival firms. But the CEO would probably have been fired for doing so, as the groups at rival firms would have continued for two more years to record "supernormal" profits and bonuses. A credit-risk officer might have hedged the firm's exposure with investments designed to underperform MBSs while the boom lasted, but to offset losses from them during the bust to follow. But that officer too would probably have been fired, for leaving two years worth of extraordinary profits on the table. Lowly portfolio managers might have purged their clients' accounts of MBSs, thereby reducing their rates of return. But some if not all of those clients would then have requested different account managers, or moved to different firms, probably costing the managers their jobs! In short, there was hardly a desk on Wall Street from which it would have been safe to act on the knowledge that the housing bubble was ready to burst, without knowing the precise date on which it would do so. In Lo's opinion, the psychology of greed makes periodic crises unavoidable. That being the case, the prudent course would seem to be to prepare for them.

Lo closed with a 14-point program of preparations worth making in anticipation of the next crisis.  His suggestion that banks and brokerages that seem too big to fail be broken up into smaller units needs no explanation. Nor do two suggestions pertaining to education that finance, economics, and risk management be taught in high school, and that a new discipline of "risk accounting" be created. Finally, his call for a small "derivatives tax" to fund university programs in financial engineering is straightforward.

Monday, November 26, 2012

Randomness in Markets and Life

Complied from works of Nassim Nicholas Taleb.

Randomness is everywhere from social science , life events or financial markets. Fat tails or wild events are probably the ones with massive impact.  Nature has designed us to fool ourselves easily , we are incompetent to understand randomness and that we are unable to know "what we do not know" . The lack of understanding of  "what we do not know"  is what makes us more prone to random events. We see hard working people on top but there is a group of people with high intellect on top so we cannot suggest which trait leads to success. Risk takers are able to capture profits and at the same time they are more prone to loss. 

Trick is to stay away from people who think they are thinkers but actually they are just mere entertainers. Journalists are somehow intellectual as they pin out the loop holes in society and are not part of herds. My principle hobby is to tease those who take themselves and their knowledge too seriously. Probability is something that fools us easily , it is not an engineering or science but it is knowledge of skepticism. It is the acceptance of ignorance and development of knowledge which we do not know.Skepticism is generally any questioning attitude towards knowledge, facts, or opinions/beliefs stated as facts and doubts regarding claims that are taken for granted elsewhere.
Hard work and discipline are part of success and not itself road to success as randomness and Black Swans can attack these and people may call you a looser or maybe someone who failed. But some state that chance plays no part in success , yet I see many not so hard working and not so intelligent people at top while few hard working geniuses failed. Risk takers are more prone to profits and failures and they usually are the confident geniuses. Black Swan can be good or bad, Google's success was a good Black Swan event while 9/11 was a bad Black Swan. Literary minds are more prone to randomness as many concepts lead to confusion and more thinking puts one to contradictions. Bad information is worse than no information as it makes us more prone to Black Swans. Small losses do not matter if profits are high, this is the code of a true risk taker but at the same time he should be educated enough to be able to estimate or catch the Black Swans as these may completely destroy him.

Never ask a guy if he is from Sparta , because if he is you will know by his habits and if he is not you are hurting him by asking him. In same way do not ask a trader how much profit he made or how mush loss he had to face it will make him sad. Accountants can never understand Randomness and Probability and even the financial movements are not to be calculated by accountants and they are book-keepers only.1990s saw the arrival of scientists in Wall Street, most of these were Phds and were mostly from Physics (Theoretical) along with Mathematics from Russia, France and China. Most of these came from Russia, Russian Physics-PHd quants were ruling the Wall Street. This was due to the reason that MBAs wanted more and in half the price firms were able to hire Russian Scientists trained under the most mathematical minds. In face Russian Physics departments were top at that time in Maths and Theoretical Physics. But the lack of knowledge in Probability is vital for me as it gives me profits but at the same time I need few to understand my theories on Randomness. More erudition in Probability and Science of Randomness will make my business go down but academically I might get more fame. Useful tools like Monte Carlo Simulation and Stochastic Models are known to many but real philosophy of randomness requires more than just models and machines. Perfection on randomness and probability requires the right information the calculation of biasness for which we need psychologists and other academics.

Learning from history is another way to forecast and understand current events. Children learn from their mistakes but I think adults suffer from same condition but though Machiavelli was smart enough to learn from history and infact wrote treatise from history and experiences, his experiences were those which he had to face due to rare randomness. Risk takers are those people who know enough through history , experiences that they are confident to put them in that random event.
Historical determinism is the word used to describe the study of history to forecast or understand randomness. Those who are good in predicting the past and good in predicting the future  so we read history to learn the mechanism of randomness. Civil servants or classical think tanks were those who were capable and meant to forecast human behaviours and social events. Journalists are ones who use Historical determination to understand the results of a random event. Wise perceive things about to happen and silence is far better.

Bell curve has 68% observation falling between standard deviation of 1 and -1 yet the intelligent brain accepts this accuracy. Experiences and statistics may have high randomness i. e I may have a bad experience with  a New Yorker but my friends may have opposite experience. Statistics can never be 100% true and those who fall in outliers or face the unexpected have their own approach as everyone prefers experience over statistics.

I like Philosophy of Karl Popper (all his ideas on epistemology, scientific methods and open society) and David Hume(especially his Epistemology and Problem of Induction) or to some extent I am obsessed with it. George Soros is the only "rich investor" who is intellectual and also a Professor type , the only money man with intellectual power. Soros learned and understood Popper and also went on to live a Popperian life. Soros spread the ideas of Popper in banking system and in society. Popper always had problems with Statisticians and especially statistical inference .He suggests that repeated events causes one to increase acceptance and hence the error of truth increases. One experiment is not enough  to get accurate result. The more data we have the more we are prone to traps.

In fact I came across Popper through Soros (Soros being my favourite investor)Popper is the man who is unworldly a man who was isolated, self focused , arrogant ,closed to outside world, independent and very bad listener and always firm on his beliefs though he helped people for good ideas and helped them to guide in their careers. Soros used more philosophy in banking than other banker, he was creative enough to apply every bit of Popper's philosophy in banking. Popper shared that Open Society is one where no permanent truth can exist and his idea was made to use by Friedrich Hayek .

What is the probability of winning New Jersey lottery twice...... 1/17 trillion. Randomness does not look random, we do know what events have high or low randomness but we are not able to catch it so in turn we are always fooled . Calculating randomness is hard for  every event (whether high or low randomness). High randomness though can either ruin you or make you billionaire but low randomness is not that massive in impact. Even when patients get the news of cancer etc their reaction is random towards it i. e some get sad and some even faint.
Brownian Random Walk used in finance has one condition that probability of success does not change with incremental step, same as Binomial model. Non-linearity and linearity is another effect that can cause drastic changes. If you try playing piano you may fail a lot of time and suddenly you start playing Chopin's sonatas .For such non-linear events we need not use Linear Statistical models as we will have large errors. Same is true for marginal benefit , the glass of water you drink after gym will have no effect and suddenly the 4th one satisfies you. 

Who exerted most influence in economics in past two centuries.... it was not Keynes, Milton Friedman , Samuelson or Marshall but they are two non-economists: Amos Tversky and Daniel Kahneman who made new ideas in Behavioural Finance/Economics. They worked on field of Heuristics and Rationality. 

The more out of the world you are i.e.  the more anarchist or rebel you are the more you are prone to randomness as you are not part of normal people and that you have less ability to predict social randomness. One great example is that of a man who is not into social media like me and so the recent music or most debated topic are not known to him and thus he faces trouble when he goes to a nightclub or even to a cafe for relaxation.  The more mathematics and Probability you know the more prone you are to randomness , this is because for events which are 2 dimensional or more are studied using sampling and other mathematical models and thus increasing error in results. Luckily the most events in a normal person's life are one dimensional. In same way if you are more emotional then it means you are less rational .Some people get angry when the car behind you gives you a loud horn when you do not act rapidly when light gets green , now 1 millisecond is not worth anything but for the back car it is something. Emotions also change your decisions accordingly , emotions might cause you to loose i. e some people leave their job if they are slightly taunted while more rational people or the less emotional think about the loss and stay calm and so are in control of their emotions and decisions in life.

But how do we know that we are going to face a rare and random event?This is the problem that needs to be perfected in order to avoid being fooled.

Sunday, November 25, 2012

Actuaries VS Quants

The following article is taken from August 2008 issue of The Actuary magazine.
Those working in the two fields of actuarial science and quantitative finance have not always been totally appreciative of each others’ skills. Actuaries have been dealing with randomness and risk in finance for centuries. Quants are the relative newcomers, with all their fancy stochastic mathematics. Rather annoyingly for actuaries, quants come along late in the game and thanks to one piece of insight in the early ‘70s completely change the face of the valuation of risk. The insight I refer to is the concept of dynamic hedging, first published by Black, Scholes and Merton in 1973. Before 1973 derivatives were being valued using the “actuarial method,” i.e. in a sense relying, as actuaries always have, on the Central Limit Theorem. Since 1973 and the publication of the famous papers, all that has been made redundant. Quants have ruled the financial roost. But this might just be the time for actuaries to fight back.   

I am putting the finishing touches to this article a few days after the first anniversary of the “day that quant died.” In early August 2007 a number of high-profile and previously successful quantitative hedge funds suffered large losses. People said that their models “just stopped working.” The year since has been occupied with a lot of soul searching by quants, how could this happen when they’ve got such incredible models?

In my view the main reason why quantitative finance is in a mess is because of complexity and obscurity. Quants are making their models increasingly complicated, in the belief that they are making improvements. This is not the case. More often than not each ‘improvement’ is a step backwards. If this were a proper hard science then there would be a reason for trying to perfect models. But finance is not a hard science, one in which you can conduct experiments for which the results are repeatable. Finance, thanks to it being underpinned by human beings and their wonderfully irrational behaviour, is forever changing. It is therefore much better to focus your attention on making the models robust and transparent rather than ever more intricate. As I mentioned in a recent blog, there is a maths sweet spot in quant finance. The models should not be too elementary so as to make it impossible to invent new structured products, but nor should they be so abstract as to be easily misunderstood by all except their inventor (and sometimes even by him), with the obvious and financially dangerous consequences. I teach on the Certificate in Quantitative Finance and in that our goal is to make quant finance practical, understandable and, above all, safe.
When banks sell a contract they do so assuming that it is going to make a profit. They use their complex models, with sophisticated numerical solutions, to come up with the perfect value. Having gone to all that effort for that contract they then throw it into the same pot as all the others and risk manage en masse. The funny thing is that they never know whether each individual contract has “washed its own face.” Sure they know whether the pot has made money, their bonus is tied to it. But each contract? It makes good sense to risk manage all contracts together but it doesn’t make sense to go to such obsessive detail in valuation when ultimately it’s the portfolio that makes money, especially when the basic models are so dodgy. The theory of quant finance and the practice diverge. Money is made by portfolios, not by individual contracts.
Imperial College London started the famous MSc Actuarial Finance , which gave birth to the new field of actuary concerned with investment banking.

Actuaries are better in accessing risk than Quants.

In other words, quants make money from the Central Limit Theorem, just like actuaries, it’s just that quants are loath to admit it! Ironic. It’s about time that actuaries got more involved in quantitative finance. They could bring some common sense back into this field. We need models which people can understand and a greater respect for risk. Actuaries and quants have complementary skill sets. What high finance needs now are precisely those skills that actuaries have, a deep understanding of statistics, an historical perspective, and a willingness to work with data.With Solvency II coming up in 2013 there will be sudden increase in the demand of actuaries.

Saturday, September 15, 2012

Physicists from Wall Street

Over the past decade, the number of Ph.D. physicists employed in the financial community has increased dramatically. Once considered something of an anomaly on Wall Street and in banking, physicists—and their fellow Ph.D.’s in mathematics, computer science, and engineering—have become a critical element to successful investment strategies, gradually replacing many employees who lack strong statistical and analytical backgrounds. Today, quantitative methods are commonplace on Wall Street, despite concerns about their predictive accuracy, and the proliferation of Ph.D. physicists in financial activities has made competition for these lucrative positions more intensive than ever.

“Investing is increasingly becoming dominated by physicists, mathematicians, electrical engineers, and programmers,” says Adrian Cooper, founder and president of Wall Street Analytics (Palo Alto, CA), where roughly one-third of the employees are Ph.D. physicists. Peter Carr, who heads the Equity Derivatives Research Group at Bank of America Securities (New York, NY), recalls that all of his interviewers for his first position at Morgan Stanley were physicists. Physicists in finance generally fall into two categories: those attempting to predict the stock market to achieve superior return, and more commonly—those who use quantitative methods to assess and manage investment risk, a group known as quantitative analysts, or “quants.” Investment banks are highly leveraged institutions, with book assets that often greatly exceed the value of the firm. Their goal is to maintain a neutral position—a balance between gainers and losers—as various assets in a portfolio rise and fall in value.Hence, “risk management is more technical than ever,” says Neil Chriss, a vice president and portfolio manager at Goldman Sachs Asset Management, who heads a fledgling master’s program in financial mathematics at New York University. “The need to control risk has become a computationally intensive problem, involving the ability to price many different assets quickly.”

Not surprisingly, the problem-solving skills of physicists are useful in this capacity, as are their abilities to view a problem in a broader context, separate small effects from larger ones, and translate intuition about how something works into formal models. “Bond traders will try to persuade you that there’s an emotional aspect that must be understood behind certain bonds, but that really isn’t the case,” says Cooper.

“A bond is a mathematical instrument which performs according to precise characteristics, and in order to analyze it properly, you need people capable of understanding the math behind those characteristics.” Although physicists have helped foster the widespread use of quantitative methods in the financial community, the revolution actually began with fundamental developments in the mathematics of finance, dating back to 1900, when Louis Bachelier introduced a Brownian motion, or “random walk,” model of price variations. 
Wall Street , US

In 1953, mathematician Harry Markowitz introduced his Nobel Prize-winning work on mean-variance analysis, which gave birth to theuse of quantitative methods for predicting the stock market. Then, in the 1960s and early 1970s, Benoit Mandelbrot—now widely known as the “father of fractals” and an IBM Fellow Emeritus at IBM’s T. J. Watson Research Center—proposed a model of price variations that eventually evolved into the concept of fractional Brownian motion in multifractal time.

Among other conclusions, Mandelbrot, who worked at IBM from 1958 to 1993, demonstrated that wealth acquired on the stock market is typically acquired during a small number of highly favorable periods—a finding markedly different from the Brownian model, which predicts small gains consistently over time.

Benoit Mandelbrot

A major turning point occurred in 1973,when economists Fischer Black and Myron Scholes devised an equation to calculate the value of options in simple derivative dealings, best described as an option to buy a stock in the future at a specified price. (The term derivative is used because the value of the contract derives from the value of the underlying stock.) The Black-Scholes approach was later extended and applied to more complex derivatives, particularly interest rate derivatives. Today, more than $14 trillion is invested in derivative securities, three times as much as is invested in the ordinary stocks and bonds from which they are derived, and the quantitative analysts trading these staggering sums include many Ph.D. physicists.

“Without the problem-solving skills of physicists, there would be a great employment shortage on Wall Street,” says Steven Shreve, a professor of mathematics at Carnegie-Mellon University, because financial institutions now use quantitative methods to hedge risk in trading derivative securities and other financial instruments. “Physicists didn’t create that fact, but they helped build the human resource needs of the banks.”

The demand for financial-modeling system has driven the formation of numerous
start-up companies, many founded by Ph.D. physicists drawn to the industry by the technical challenges and potential monetary rewards. (Base salaries on Wall Street can be as much as three times that of traditional physics positions.) Cooper earned his Ph.D. in theoretical physics from Stanford University, but found himself comparing the career satisfaction and financial rewards of Ph.D.’s his age who had followed the traditional career path with those who had gone into finance. “It was pretty clear which direction was more appealing,” he says. Cooper went on to found Wall Street Analytics, which develops software for modeling financial systems for mortgage-pool investments.

Nigel Goldenfeld, professor of physics at the University of Illinois, Urbana-Champaign (UIUC), earned his Ph.D. from the University of Cambridge in England and specializes in statistical, theoretical, and computational physics. His first Ph.D. student at UIUC ended up working for Goldman Sachs, which sparked Goldenfeld’s interest in the physics of finance. Convinced he could improve on the calculation techniques used, he founded NumeriX in 1996 with fellow physicists Alexander Sokol and Mitchell Feigenbaum and entrepreneur Michael Goodkin

Nigel Goldenfeld

Norman Packard
York-based venture that markets fast numerical software products for derivative-risk management. Physicists attempting to predict the stock market look for patterns in the data of stock prices and foreign-exchange markets. The Prediction Company (PC), based in Santa Fe, New Mexico, is perhaps the best-known company focusing on this sector of finance. The company develops advanced forecasting technologies for prediction and computerized trading of financial instruments, based on the assumption that the stock market is not completely random but has short-term pockets of predictability. “Our task for models is to detect the mispricing of an asset, and make a trade based on [that],” says Norman Packard, one of PC’s founders.

 Along with fellow high-energy physicist Doyne Farmer, Packard developed a computerized system for beating the roulette wheel in the 1970s based on the then emerging field of chaos theory. They subsequentlym sold it to other entrepreneurs for further development. The pair then concluded that financial markets offered another example of a complex system that might be amenable to predictive technology. They founded PC in 1991, and within a year, the company had signed an exclusive agreement to provide predictive signals and automated trading systems to O’Connor and Associates (now part of Swiss Bank), a highly successful Chicago-based trading firm that had made millions in derivatives trading using the Black-Scholes equation.

J. Doyne Farmer

The skeptics, however, still remain unconvinced. “If they could do it, they wouldn’t be wasting their time with a company. They would just be sitting there buying and selling IBM share options,” Cooper says. Prevailing attitudes among academics toward physicists working on Wall Street have changed in the last decade. Emanuel Derman, who earned his Ph.D. in physics from Columbia University in the 1970s, is now a managing director at Goldman Sachs and head of its Quantitative Strategies Group. He finds that the financial world is no longer viewed as a second-rate “alternative” career for physicists unable to obtain positions in academia and industry. Instead, finance now is a highly desirable first choice, as evidenced by the number of tenured professors who have left their academic positions for more lucrative careers in finance. Ironically, physicists interested in pursuing careers in finance today may have become victims of their predecessors’ success. “It used to be a gravy train for physicists and other mathematically oriented people, but now the job market has become saturated,” says Carr.

Emanuel Derman

 Few financial houses are hiring additional quants, and today the ability to solve differential equations and perform basic programming isn’t enough to land a job on Wall Street. New Ph.D. physicists also need a basic understanding of options, pricing theory, interest rate theory, and other foundations of finance to be considered. “The stakes are higher than they’ve ever been in terms of the level of knowledge expected for entry-level [quant] positions,” says Chriss.

This increased competitiveness has spurred the formation of numerous master’s degree programs in computational finance (also called financial engineering or mathematics in finance) at institutions around the country, i.e Carnegie Mellon University.

 Financial data are also wildly complicated, yet many of their features are reproduced by Beniot Mandelbrot’s multifractal model of price variation, which is also of surprising simplicity. Benoit pioneered the concept in 1994 with a 12-month program combining coursework from four separate academic departments: mathematical sciences, statistics, computer science, and business.

Other schools have followed suit, including Purdue University, the Massachusetts Institute of Technology, Columbia University, Cornell University, the University of Michigan, the University of Chicago, and New York University. At UIUC, students are able to complete an accelerated master’s in finance program in conjunction with their physics Ph.D.’s. Although he supports the rationale for master’s programs in computational finance, Cooper argues the need to preserve the traditional focus of university physics programs.His primary concern is that the growing emphasis on finance as a career option for physicists could undermine graduate education and turn Ph.D. candidates into mortgage traders too early in their development.

Several universities are mulling the possibility of running physics and finance graduate programs in tandem. “These are the universities that have been entrusted to pass on the learning that has taken generations to amass in physics, mathematics, and other subjects,” Cooper says. “If the physics departments aren’t protecting the subject of physics, who is going to do it?” But Packard believes the threat of a “brain drain” is not limited to finance. “Financial markets are the least of physics’ worries,” he says. “The field is facing an even more severe brain drain from a number of other areas, such as electronics and computer engineering.”

As successful as today’s financial models have been, there remains substantial room for improvement. Although pleased at the increased trust placed in quantitative methods by the financial community, Thierry Kaufmann, a theoretical physicist who heads Purdue’s computational finance master’s program, admits that existing models are not 100% accurate, which poses serious potential consequences. “Sometimes excessive trust is placed in these quantitative results by people who lack the proper background and make very risky decisions based on them,” he says. “They can lose a lot of money.”

Econophysics, also known as the physics of finance, is the study of the dynamical behavior of financial and economic markets. Recently, a vast amount of market data has become available allowing empirical studies of market behavior to be performed.

Although the models used for equity derivatives have proven fairly robust, many surprises still take place in markets heavily dependent on bonds and interest-rate movements and in volatile foreign-exchange markets such as Indonesia. Part of the problem, says Goldenfeld, is that many models were created with an eye to being easily calculable.

Hence, these models are not faithful to the complexities of real market dynamics. “It’s no good having people help you calculate if you’re using the wrong model,” he says. “You’re getting the wrong answer faster and more accurately. You want to be able to get the right answer fast and accurately.” Among the sharpest critics has been Mandelbrot himself, now Abraham Robinson Professor of Mathematical Sciences at Yale University, who believes that the current models seriously underestimate the frequency of large fluctuations in stock value. For example, Alcatel, a French telecommunications equipment manufacturer, experienced severe volatility in its stock prices last year, which fell 40% in one day, fell another 3% over the next three days, and then rebounded by 10% on the fourth day. “The classical financial models used for most of this century predict that such ‘10 sigma’ precipitous events should never happen,” Mandelbrot says, with estimated probabilities of a few millionths of a millionth of a millionth of a millionth. In reality, such spikes occur quite regularly—as often as every month—with probabilities closer to a few hundredths. Far from varying continuously, as such models tend to assume, prices oscillate wildly, often discontinuously, at all time scales “Volatility, far from being a static entity to be ignored or easily compensated for, is at the heart of what goes on in financial markets, Mandelbrot concludes. 

Underestimating the frequency of 10 sigma events, a technical term for enormous price fluctuations, can have serious global economic implications, as evidenced by last year’s fears of damage to financial markets worldwide brought on by heavily leveraged trading by a hedge fund called Long Term Capital Management.This potential threat was narrowly averted by a bailout of the hedge fund paid for by major investment houses. Even academic physicists are beginning to look more critically at some of the prevailing modeling assumptions used by the financial community. “It’s becoming its own discipline,” says Derman. In fact, financial markets provide an excellent practical field of study for those interested in the behavior of complex and nonequilibrium dynamical systems because there is a wealth of data available. “Previous attempts to look at complex systems, in my view, have not been successful because people have operated at a level of generalities rather than rolling up their sleeves and doing honest spadework,” says Goldenfeld. “What’s happening now is exactly what I had hoped: people are digging in and trying to understand the financial markets from a physicist’s perspective rather than that of a financial economist.” However, he remains a strong proponent of the value of firsthand experience with financial markets when studying such systems. “Thermodynamics was invented by engineers who wanted to make steam engines, not by people thinking about quantum states and other abstract concepts,”
Goldenfeld says.