Why the latest SWS survey result is questionable — statistics say

148 views

A number of polls demonstrate a widening disparity in net satisfaction ratings of Duterte’s government between an SWS survey and internet polling, with one producing only “good” net satisfaction rating and the second reporting that the net satisfaction rating is “excellent.”

The gap in results between the latest SWS and online surveys has been a talk online, with SWS poll putting President Duterte’s government to a “good” net satisfaction rating, while internet surveys point to “excellent” rating.

But the contrast in the two types of surveys is particularly put in a limelight, because they were not conducted concurrently. Vote adjustment methodologies are somewhat similar (but adjustments will be considered in our discussion below to remove trolls or fake accounts, as claimed by the opposition, participating in the online surveys). In the interview poll of Social Weather Stations, the poll was conducted from September 23-27, 2017 and found out that 67% of adult Filipinos satisfied, 14% undecided, and 19% dissatisfied with President Rodrigo R. Duterte’s performance. Once the “undecided” is excluded, a net satisfaction rating of +48 is obtained, which is 18 points lower than the “very good” net satisfaction rating of +66 in June 2017. Online surveys were conducted just this week upon hearing of the drop of President Duterte’s net satisfaction rating.

Though the two types of surveys were not done concurrently, they were still made some weeks after the deaths of teenagers Kian Loyd de los Santos, Carl Angelo Arnaiz, and Reynaldo de Guzman, which others claimed that the deaths sparked nationwide outcry against extrajudicial killings and the rising death toll in Duterte’s war against drugs, hence a decrease in net satisfaction and trust ratings.

SWS vs. Raffy Tulfo in Action in Net Trust Ratings

The online survey regarding trust in Duterte’s government by Raffy Tulfo in Action garnered a total of not less than 129,000 engagements with 89,000 hearts (Yes) and 4,500 angry faces (No). Other emoticons were not considered in the analysis, which resulted to 93,500 valid votes.

Raffy Tulfo in Action (Net Trust Rating)
Survey results of Raffy Tulfo in Action regarding Trust in Duterte’s government. Check Raffy Tulfo in Action post.

The September 2017 survey by SWS found 73% of Filipinos with much trust, 15% undecided, and 12% with little trust in Pres. Duterte, which gives a net trust rating of +60, classified by SWS as “very good”. The result is a 15-point decline, and one grade below, the excellent +75 (82% much trust, 7% little trust) in June 2017. With the online poll by Raffy Tulfo in Action, conducted since October 10 (Tuesday), by contrast, the net trust rating is 31-point above SWS survey [Chart 1]. With the adjusted total votes for “yes” — minus 10,000 considered online trolls or fake accounts participating the poll who voted “yes,” — the result still very much higher than the Social Weather Stations’ face-to-face interview of 1,500 adults (18 years old and above) nationwide: 600 in Balance Luzon, and 300 each in Metro Manila, Visayas, and Mindanao (sampling error margins of ±2.5% for national percentages, ±4% for Balance Luzon, and ±6% each for Metro Manila, Visayas, and Mindanao), above 29 points [Chart 2]. Raffy Tulfo in Action online poll shows that the net trust rating of President Rodrigo Roa Duterte in any of the actual and adjusted votes is “excellent“, based on the SWS terminology for net satisfaction and net trust ratings.

SWS vs. Raffy Tulfo in Action Online Polls (Edited)
Chart 1 is SWS vs. Raffy Tulfo in Action (actual) in Net Trust Ratings. Chart 2 is SWS vs. Raffy Tulfo in Action (adjusted) in Net Trust Ratings, where a number of online trolls and Filipinos outside the country of 10,000 was considered.

Meanwhile, the maximum number of “yes” votes deduction to get the minimum “excellent” rating of +70 is 60,070 [Chart 3]. That’s a huge number and many, including popular journalist Raffy Tulfo, would doubt that the number of online trolls, fake accounts, or Filipinos answering outside the Philippines who participated in the online survey was that high. Or are they the Filipinos who participated in the online poll who do not accurately represent the Philippine population being surveyed (18 years old and above) as a whole? One can surely doubt.

SWS vs. Raffy Tulfo in Action Online Polls (with Sample Errors)
Chart 3 is SWS vs. Raffy Tulfo in Action Online Polls (Deduction of 64,070 who voted “yes”). Table 1 is Sample Errors of SWS and Raffy Tulfo in Action in Net Satisfaction Ratings with estimated population of 50 million aged 18 years old and above.

The two polls in the analysis show huge disparity in net trust rating, which is questionable. SWS survey (with sample error of ±2.5% at 95% confidence interval and sample proportion of 50%-50%) result does not conform with any of the results using sample sizes at 95% or 99% confidence interval and sample proportions of 50%-50%, 10%-90%, or 90%-10% [Table 1], see sample error calculator, in Raffy Tulfo in Action online poll. Sample proportion is the proportion of people in the population being surveyed who are expected to answer a certain way on the key measure in the survey. In our analysis, we adjusted it up to 10% who have “much trust” and 90% who have “little trust” in Duterte’s government, following the issues of the deaths of teenagers Kian Loyd de los Santos, Carl Angelo Arnaiz, and Reynaldo de Guzman. Scientifically, result of sample with larger sample error should follow the result of sample with very small sampler error, in the case of RT in Action, ranging from ±0.2% to ±0.8%. Otherwise, there’s a problem with SWS survey distribution of face-to-face interviewees. But wait. Maybe this is just an isolated case and need not to be given emphasis. But are we assured that this is just this case?

Let’s look at the other online poll results.

SWS vs. The Filipino News in Net Satisfaction Ratings

For another comparison to SWS survey result, we used the online voting intention that lasted in 4 hours conducted this week by The Filipino News in its Facebook page. The online poll results show that 41,430 voters are “satisfied” and 936 “dissatisfied” with President Duterte’s government.

The Filipino News Satisfaction Survey

The figures below, which some [Chart 5, Chart 6] are adjusted on the same principles as with the SWS face-to-face interview survey with much less dramatic effect, show: President Duterte has +95.6 net satisfaction rating [Chart 4] with the actual votes; and +94.2 net satisfaction rating with the first adjustment of 10,000 votes who were considered as online trolls, fake accounts, or Filipinos answering from outside the Philippines. SWS survey regarding satisfaction and trust ratings mainly focuses on qualified residents currently living in the Philippines. These, +95.6 and +94.2, net satisfaction ratings are both “excellent,” based on the SWS terminology for net satisfaction and net trust ratings.

The Third Quarter 2017 Social Weather Survey, conducted from September 23-27, 2017, found 67% of adult Filipinos satisfied, 14% undecided, and 19% dissatisfied with President Rodrigo R. Duterte’s performance, which resulted to a net satisfaction rating of +48 (% satisfied minus % dissatisfied), classified by SWS as good. This is 18 points below the very good +66 (78% satisfied, 12% dissatisfied) in June 2017.

SWS vs. The Filino News Online Poll Comparative Analysis
Chart 4 is SWS vs. in Net Satisfaction Ratings. Chart 5 is SWS vs. The Filipino News (adj. -10,000 who voted “satisfied”) in Net Satisfaction Ratings. Chart 5 is SWS vs. The Filipino News (adj. -16,501 who voted “satisfied”) in Net Satisfaction Ratings.

With calculation of maximum number of “satisfied” votes deduction; 16,501; to get the minimum “excellent” net satisfaction rating, we concluded that there could be biases running in SWS survey on the question, possibly on the distribution of interviewees across the country.

Sample Errors of SWS and The Filipino News in Net Satisfaction Ratings
Table 2 is Sample Errors of SWS and The Filipino News in Net Satisfaction Ratings.

Table 2 shows the sample errors of SWS and The Filipino News in net satisfaction ratings. As displayed, all sample sizes of The Filino News at 95% or 99% confidence intervals and sample proportions of 50%-50%, 10%-90%, or 90%-10% demonstrate very small sample errors ranging from ±0.3% to ±0.7%.

Summary

Results of SWS face-to-face interview survey with smaller sample size of 1,500 do not follow results of online polls with as much as 29,000 voters. Scientifically, result of sample with larger sample error should follow the result of sample with very small sampler error, which entails that there could be biases with SWS survey distribution of face-to-face interviewees.

Both types of poll were conducted some weeks after the deaths of teenagers Kian Loyd de los Santos, Carl Angelo Arnaiz, and Reynaldo de Guzman, which others claimed that their deaths affected the net satisfaction and trust ratings of President Duterte, but they yielded different results.

There is no definitive way of adjudicating SWS poll, but as good a guess as any is that the right answer lies somewhere in between studying and asking again the interviewees being surveyed by them and relying on the results of larger sample size, in this case, the results of Raffy Tulfo in Action and The Filipino News online polls.

comments