Photo by Simon Bak on Unsplash

Brick-N-Mortar vs E-Commerce : Opinion Mining

Opinion mining as a business analysis tool can help businesses steer their resources in a positive direction. Opinion mining through Twitter is a popular way to gather public sentiment regarding politics, and social issues. In this project I try to capture public opinion on brick-n-mortar stores versus online shopping. With the continued rise in popularity of the world wide web, electronic transactions, less time availability from people, recommendation engines, the landscape of the consumer-retail relationship is dynamically shifting. Proponents of the shift to e-commerce held that consumers are looking for an experience and a feeling when they enter a physical space, let alone a past time. Others argue that e-commerce is convenient and time efficient.

In this project I try to answer all these questions as well as consumer sentiment during the holidays, pre-covid and post-covid situations, the future of retail, and the pros and drawbacks of each as stated by popular opinion.

The first step in these answers is gathering the data. I use tweets as a sample to gather public opinion of brick-n-mortar stores and online e-commerce. So I use Twint to web scrape Twitter with the following code on the OS command prompt.

import twint
twint -s #ecommerce --since '2015-01-01' -o ecommerce.csv --csv
twint -s #bricknmortar --since '2015-01-01' -o bricknmortar.csv --csv

With the data gathered, I constructed graphs indicating word frequencies and sentiments of the search terms. I should also note that a lot more data was available for e-commerce than brick-n-mortar. About 120,000 tweets compared to 2 million. But note relative comparisons will be made rather than hard numbers.

I noted the word frequencies of brick-n-mortar and e-commerce and listed some of them below.

My next step was to divide the sentiment of the tweets into positive, neutral, and negative. From there, I took the word frequencies of the most common words found in each sentiment to try to answer some questions about public opinion about e-commerce and brick-n-mortar shops.

I then separated the tweets by years to see if there was a growing or shrinking trend in the two retail types within the past 5 years. It should be noted that 2020 was the year of covid and data suggesting growth and shrinkage for retail type must have the pandemic in mind.

I then separated the tweets into the positive, neutral, negative sentiment noting the proportion of the three sentiments in e-commerce and brick-n-mortar. This shows the general consumer confidence and outlook of the retail type and how they feel about brick-n-mortars and e-commerce. There seems to be a low negative opinion on both store types.

In summary, our findings show that e-commerce has a very positive acceptance among retailers whereas brick-n-mortar has a relatively more neutral opinion.

Did these opinions change during the holidays?

There does not seem to be any major shift in opinion during the holiday shopping season (Black Friday — Christmas). But a slightly more positive sentiment toward brick-n-mortar stores seems to have accrued during holiday season.

And that is all there is to opinion mining of brick-n-mortar stores and e-commerce. Further studies I would like to conduct are : Are certain products better marketed and profitable under a brick-n-mortar vs. the e-commerce model? Which industries or products is this true for? For example, would consumers be comfortable with online shopping for cars? One can expect that test-driving a car would be a priority on the consumer’s list.

Would shopping in the future mean more than just buying? Would sellers provide extra incentives and experiences in the shopping experience. Does online shopping work in the culture of instant gratification. And moreover, what does the future hold for post-covid society where jobs continue to be remote and therefore lead to lifestyle changes for the consumer. What will happen to malls, large corporate retailers vs small independent retailers who can offer similar products without paying for overhead.

These are all data science questions but to a much larger extent questions about our future lives, preferences, and global economy.

Thank you for reading.