Since 2007, California has had one of the most aggressive incentive programs in the country for putting solar-electric panels on the rooftops of homes and businesses. Its $2.2 billion California Solar Initiative (CSI)has provided a per-watt rebate for installing residential and commercial photovoltaic systems.

During this period, the solar industry in the state has experienced double-digit growth and to date has installed more than 245,000 systems capable of producing 2,365 megawatts of electricity.

As a result, CSI has been widely touted as a major success. However, a new study that uses an advanced method called "data-driven agent-based modeling" finds that using the same amount of money in a carefully optimized program to provide systems to low-income households at little or no cost would have done more to stimulate the adoption of rooftop solar systems than the incentive-based approach that the initiative used.

The new study was performed by doctoral student Haifeng Zhang and Assistant Professor of Computer Science Yevgeniy Vorobeychik at Vanderbilt University in collaboration with Joshua Letchford and Kiran Lakkaraju at Sandia National Laboratories and is described in the Proceedings of the 14th International Conference on Autonomous Agents and Multiagent Systems scheduled for publication in May.

The problem with assessing the success of programs like CSI is that you need to compare them with what would have happened in the same world without the incentives.

"The problem with assessing the success of programs like CSI is that you need to compare them with what would have happened in the same world without the incentives," said Vorobeychik. "Of course, that is impossible, so you need to use techniques, like the our data-driven agent-based modeling methodology, that analyze the data and make predictions on how it would behave under different circumstances."

Vorobeychik and his colleagues used CSI data on rooftop solar installations from May 2007 to April 2013 in San Diego County for their analysis. This included detailed information on 8,500 projects, including system size, reported cost, incentive amount, whether system was purchased or leased, installation date, location and acreage of the lot and size of the home.

Using the data from May 2007 to April 2011, the analysts applied methods developed in the field of machine learning to create a model of the relative importance that two factors - economic benefit and peer influence - play on an individual's decision to install a rooftop solar system.

In this case, they used the number of existing installations in an area as the measure of peer influence. Then they tested the accuracy of their model by predicting the pattern of adoption from May 2011 to April 2013.

To test the effectiveness of financial subsidies on the market, the researchers looked at a range of incentives from zero to eight times the actual the level. They were surprised to find very little difference in adoption rates between no subsidy and the actual subsidy. Increasing the subsidy rate did increase the number of adoptees but even at the highest level the effect was extremely modest.

"Despite the fact that policy makers have paid overwhelming attention to increasing the economic benefits by reducing costs to get people to adopt solar technology, our analysis indicates that this has much less effect than generally perceived," said Vorobeychik. "That is what the data is telling us."

One of the factors that reduced the impact of the incentive program, they found, was the increasing prevalence of leasing. When the program started, homeowners were buying virtually all of the systems so they were receiving the rebates. In recent years, however, the installers began offering attractive leasing packages that have become increasingly popular. When systems are leased, however, it is the installers who receive the rebates and, according to the data, they have not been passing the incentive along to the homeowners.

Since the financial incentives had such a limited effect, the researchers decided to see if they could design a more effective subsidy scheme based on peer influence: The fact that the larger the number of solar installations in a person's neighborhood the more likely they are to decide to install a system.

To do so, Vorobeychik and his colleagues explored the impact of scaling up a small existing CSI program that gives solar systems to low-income families for little or no cost. They determined that an optimal program of this type could result in significantly greater adoption compared to the incentive approach. In addition, they found that its relative advantage increased at higher budget levels.

The finding is consistent with the academic literature that provides strong evidence of the importance of peer influence. "But there has not been as much attention paid to peer effects at the policy level, perhaps because policy makers are used to the idea of stimulating markets by offering subsidies and because there is less understanding about how to leverage peer influence," Vorobeychik said.

In the past, people have used traditional econometric techniques to evaluate programs of this kind. The issue with such traditional analysis is that it does not take into account the complex ways in which the individual decisions made by a diverse cast of characters interact. As a result, the associated models cannot easily be used to evaluate policies that leverage individual diversity (such as giving systems away to low-income households).

According to the computer scientist, that is where data-driven agent-based modeling has an advantage. It is a relatively new technique made possible by recent increases in computer power and the growing availability of large amounts of digital data that has become the favored approach in studying emergent behaviors that arise from the actions of large numbers of agents in social and economic contexts.