Araştırma Makalesi
BibTex RIS Kaynak Göster

The Effect of Asynchronous Message Queues on the Communication of IoT Devices

Yıl 2021, Cilt: 4 Sayı: 3, 44 - 53, 30.12.2021

Öz

Nowadays, IoT (Internet of Things) devices have reached quite high numbers. This situation brought with it the presence of high density, unstructured data. The difficulties in obtaining, processing, storing and visualizing this data made it necessary to use the components of the big data system. High density data should be taken from IoT devices instantly, meaningful data should be obtained from these unstructured data, the meaningful data should be stored and presented at the request of the user when needed. In this article, a model is designed to process the data obtained from IoT devices and transmit them instantly to the end user. In the study, unstructured data collected primarily from IoT devices were subjected to data pre-processing steps. Significant words were determined from the data obtained after the data pre-processing steps. For this purpose, the Helmholtz Principle has been applied. After meaningful word detection, it is directed to both Rabbit MQ messaging queue and IBM MQ message queue separately to instantly process data on the subject of each meaningful word. Apache Storm topology was used to instantly receive and process the messages transmitted to the queues. According to the results obtained, messages sent to the IBM MQ message queue are consumed 30% faster than the Rabbit MQ message queue.

Kaynakça

  • Lausch, Angela & Schmidt, Andreas & Tischendorf, Lutz. (2015). Data mining and linked open data – New perspectives for data analysis in environmental research. Ecological Modelling. 295. 5-17. 10.1016/j.ecolmodel.2014.09.018.
  • Shaqiri, Bledi. (2017). Exploring Techniques of Improving Security and Privacy in Big Data. 10.13140/RG.2.2.23201.10089.
  • S. G. Manikandan and S. Ravi, "Big Data Analysis Using Apache Hadoop," 2014 International Conference on IT Convergence and Security (ICITCS), Beijing, 2014, pp. 1-4, doi: 10.1109/ICITCS.2014.7021746.
  • M. Sogodekar, S. Pandey, I. Tupkari and A. Manekar, "Big data analytics: hadoop and tools," 2016 IEEE Bombay Section Symposium (IBSS), Baramati, 2016, pp. 1-6, doi: 10.1109/IBSS.2016.7940204.
  • K. Singh and R. Kaur, "Hadoop: Addressing challenges of Big Data," 2014 IEEE International Advance Computing Conference (IACC), Gurgaon, 2014, pp. 686-689, doi: 10.1109/IAdCC.2014.6779407.
  • A. Verma, A. H. Mansuri and N. Jain, "Big data management processing with Hadoop MapReduce and spark technology: A comparison," 2016 Symposium on Colossal Data Analysis and Networking (CDAN), Indore, 2016, pp. 1-4, doi: 10.1109/CDAN.2016.7570891.
  • S. Sagiroglu and D. Sinanc, "Big data: A review," 2013 International Conference on Collaboration Technologies and Systems (CTS), San Diego, CA, 2013, pp. 42-47, doi: 10.1109/CTS.2013.6567202.
  • G. Harerimana, B. Jang, J. W. Kim and H. K. Park, "Health Big Data Analytics: A Technology Survey," in IEEE Access, vol. 6, pp. 65661-65678, 2018, doi: 10.1109/ACCESS.2018.2878254.
  • A. R. Reddy and P. S. Kumar, "Predictive Big Data Analytics in Healthcare," 2016 Second International Conference on Computational Intelligence & Communication Technology (CICT), Ghaziabad, 2016, pp. 623-626, doi: 10.1109/CICT.2016.129.
  • J. Liao, X. Zhuang, R. Fan and X. Peng, "Toward a General Distributed Messaging Framework for Online Transaction Processing Applications," in IEEE Access, vol. 5, pp. 18166-18178, 2017, doi: 10.1109/ACCESS.2017.2717930.
  • F. Famili, W. Shen, R. Weber and E. Simoudis, “Data preprocessing and intelligent data analysis,” Intell. Data Anal, vol. 1(4), pp. 3-23, 1997. https://doi.org/10.1016/S1088-467X (98)00007-9
  • V. Agarwal, “Research on data preprocessing and categorization technique for smartphone review analysis,” International Journal of Computer Applications, vol. 131(4), pp. 30-36, 2015. https://www.doi.org/10.5120/ijca2015907309
  • B. Dadachev, A. Balinsky, H. Balinsky and S. Simske, “On the Helmholtz Principle for data mining,” Third International Conference on Emerging Security Technologies, pp. 99-102, 2012. https://www.doi.org/10.1109/EST.2012.11
  • S. Jabri, A. Dahbi, T. Gadi and A. Bassir, “Ranking of text documents using TF-IDF weighting and association rules mining,” 2018 4th International Conference on Optimization and Applications (ICOA), pp. 1-6, 2018. https://www.doi.org/10.1109/ICOA.2018.837057
  • A.G. Jivani, “A comparative study of stemming algorithms,” Int. J. Comp. Tech. Appl, vol. 2(6), pp. 1930-1938, 2011.
  • Caner Tosun. “RabbitMQ Nedir? Windows Üzerinde Kurulumu”. http://www.canertosuner.com/post/rabbitmq-nedir-windows-uzerinde-kurulumu (18.06.2020).
  • Oguzhan İnan. “Hadoop Ekosistemi ve Kullanılan Araçlar”. https://oguzhaninan.gitlab.io/Hadoop-Ekosistemi-ve-Kullanilan-Araclar/#what-is-storm (18.06.2020).
  • Womaneng. “Kafka-Flink-Storm-Platformları”. https://womaneng.com/kafka-flink-storm-platformlari/ (18.06.2020).
  • Büyükveri. “Büyük Veri Ekosistemi”. http://www.buyukveri.co/buyuk-veri-ekosistemi/ (18.06.2020).
  • Devnot. “Bir Bakışta ElasticSearch”. http://devnot.com/2017/bir-bakista-elasticsearch/ (18.06.2020).
  • Elastic. “Elasticsearch and Kibana Deployments on Azure”. https://www.elastic.co/blog/elasticsearch-and-kibana-deployments-on-azure
  • Burcu Altınok. “REST API Nedir?” https://burcualtinok.com.tr/blog/rest-api-nedir/ (18.06.2020).
  • Devnot. “REST Mimarisi ve RESTful Servisler”. http://devnot.com/2016/rest-mimarisi-ve-restful-servisler/(18.06.2020).
  • Deniz İrgin. “REST ve RESTful Web Servis Kavramı”. https://denizirgin.com/rest-ve-restful-web-servis-kavram%C4%B1-30bc4400b9e0/(18.06.2020).

The Effect of Asynchronous Message Queues on the Communication of IoT Devices

Yıl 2021, Cilt: 4 Sayı: 3, 44 - 53, 30.12.2021

Öz

Nowadays, IoT (Internet of Things) devices have reached quite high numbers. This situation brought with it the presence of high density, unstructured data. The difficulties in obtaining, processing, storing and visualizing this data made it necessary to use the components of the big data system. High density data should be taken from IoT devices instantly, meaningful data should be obtained from these unstructured data, the meaningful data should be stored and presented at the request of the user when needed. In this article, a model is designed to process the data obtained from IoT devices and transmit them instantly to the end user. In the study, unstructured data collected primarily from IoT devices were subjected to data pre-processing steps. Significant words were determined from the data obtained after the data pre-processing steps. For this purpose, the Helmholtz Principle has been applied. After meaningful word detection, it is directed to both Rabbit MQ messaging queue and IBM MQ message queue separately to instantly process data on the subject of each meaningful word. Apache Storm topology was used to instantly receive and process the messages transmitted to the queues. According to the results obtained, messages sent to the IBM MQ message queue are consumed 30% faster than the Rabbit MQ message queue.

Kaynakça

  • Lausch, Angela & Schmidt, Andreas & Tischendorf, Lutz. (2015). Data mining and linked open data – New perspectives for data analysis in environmental research. Ecological Modelling. 295. 5-17. 10.1016/j.ecolmodel.2014.09.018.
  • Shaqiri, Bledi. (2017). Exploring Techniques of Improving Security and Privacy in Big Data. 10.13140/RG.2.2.23201.10089.
  • S. G. Manikandan and S. Ravi, "Big Data Analysis Using Apache Hadoop," 2014 International Conference on IT Convergence and Security (ICITCS), Beijing, 2014, pp. 1-4, doi: 10.1109/ICITCS.2014.7021746.
  • M. Sogodekar, S. Pandey, I. Tupkari and A. Manekar, "Big data analytics: hadoop and tools," 2016 IEEE Bombay Section Symposium (IBSS), Baramati, 2016, pp. 1-6, doi: 10.1109/IBSS.2016.7940204.
  • K. Singh and R. Kaur, "Hadoop: Addressing challenges of Big Data," 2014 IEEE International Advance Computing Conference (IACC), Gurgaon, 2014, pp. 686-689, doi: 10.1109/IAdCC.2014.6779407.
  • A. Verma, A. H. Mansuri and N. Jain, "Big data management processing with Hadoop MapReduce and spark technology: A comparison," 2016 Symposium on Colossal Data Analysis and Networking (CDAN), Indore, 2016, pp. 1-4, doi: 10.1109/CDAN.2016.7570891.
  • S. Sagiroglu and D. Sinanc, "Big data: A review," 2013 International Conference on Collaboration Technologies and Systems (CTS), San Diego, CA, 2013, pp. 42-47, doi: 10.1109/CTS.2013.6567202.
  • G. Harerimana, B. Jang, J. W. Kim and H. K. Park, "Health Big Data Analytics: A Technology Survey," in IEEE Access, vol. 6, pp. 65661-65678, 2018, doi: 10.1109/ACCESS.2018.2878254.
  • A. R. Reddy and P. S. Kumar, "Predictive Big Data Analytics in Healthcare," 2016 Second International Conference on Computational Intelligence & Communication Technology (CICT), Ghaziabad, 2016, pp. 623-626, doi: 10.1109/CICT.2016.129.
  • J. Liao, X. Zhuang, R. Fan and X. Peng, "Toward a General Distributed Messaging Framework for Online Transaction Processing Applications," in IEEE Access, vol. 5, pp. 18166-18178, 2017, doi: 10.1109/ACCESS.2017.2717930.
  • F. Famili, W. Shen, R. Weber and E. Simoudis, “Data preprocessing and intelligent data analysis,” Intell. Data Anal, vol. 1(4), pp. 3-23, 1997. https://doi.org/10.1016/S1088-467X (98)00007-9
  • V. Agarwal, “Research on data preprocessing and categorization technique for smartphone review analysis,” International Journal of Computer Applications, vol. 131(4), pp. 30-36, 2015. https://www.doi.org/10.5120/ijca2015907309
  • B. Dadachev, A. Balinsky, H. Balinsky and S. Simske, “On the Helmholtz Principle for data mining,” Third International Conference on Emerging Security Technologies, pp. 99-102, 2012. https://www.doi.org/10.1109/EST.2012.11
  • S. Jabri, A. Dahbi, T. Gadi and A. Bassir, “Ranking of text documents using TF-IDF weighting and association rules mining,” 2018 4th International Conference on Optimization and Applications (ICOA), pp. 1-6, 2018. https://www.doi.org/10.1109/ICOA.2018.837057
  • A.G. Jivani, “A comparative study of stemming algorithms,” Int. J. Comp. Tech. Appl, vol. 2(6), pp. 1930-1938, 2011.
  • Caner Tosun. “RabbitMQ Nedir? Windows Üzerinde Kurulumu”. http://www.canertosuner.com/post/rabbitmq-nedir-windows-uzerinde-kurulumu (18.06.2020).
  • Oguzhan İnan. “Hadoop Ekosistemi ve Kullanılan Araçlar”. https://oguzhaninan.gitlab.io/Hadoop-Ekosistemi-ve-Kullanilan-Araclar/#what-is-storm (18.06.2020).
  • Womaneng. “Kafka-Flink-Storm-Platformları”. https://womaneng.com/kafka-flink-storm-platformlari/ (18.06.2020).
  • Büyükveri. “Büyük Veri Ekosistemi”. http://www.buyukveri.co/buyuk-veri-ekosistemi/ (18.06.2020).
  • Devnot. “Bir Bakışta ElasticSearch”. http://devnot.com/2017/bir-bakista-elasticsearch/ (18.06.2020).
  • Elastic. “Elasticsearch and Kibana Deployments on Azure”. https://www.elastic.co/blog/elasticsearch-and-kibana-deployments-on-azure
  • Burcu Altınok. “REST API Nedir?” https://burcualtinok.com.tr/blog/rest-api-nedir/ (18.06.2020).
  • Devnot. “REST Mimarisi ve RESTful Servisler”. http://devnot.com/2016/rest-mimarisi-ve-restful-servisler/(18.06.2020).
  • Deniz İrgin. “REST ve RESTful Web Servis Kavramı”. https://denizirgin.com/rest-ve-restful-web-servis-kavram%C4%B1-30bc4400b9e0/(18.06.2020).
Toplam 24 adet kaynakça vardır.

Ayrıntılar

Birincil Dil İngilizce
Konular Mühendislik
Bölüm Makaleler
Yazarlar

Ahmet Toprak

Abdül Halim Zaim

Feyzanur Sağlam Toprak Bu kişi benim

Yayımlanma Tarihi 30 Aralık 2021
Yayımlandığı Sayı Yıl 2021 Cilt: 4 Sayı: 3

Kaynak Göster

APA Toprak, A., Zaim, A. H., & Sağlam Toprak, F. (2021). The Effect of Asynchronous Message Queues on the Communication of IoT Devices. Veri Bilimi, 4(3), 44-53.



Dergimizin Tarandığı Dizinler (İndeksler)


Academic Resource Index

logo.png

journalseeker.researchbib.com

Google Scholar

scholar_logo_64dp.png

ASOS Index

asos-index.png

Rooting Index

logo.png

www.rootindexing.com

The JournalTOCs Index

journal-tocs-logo.jpg?w=584

www.journaltocs.ac.uk

General Impact Factor (GIF) Index

images?q=tbn%3AANd9GcQ0CrEQm4bHBnwh4XJv9I3ZCdHgQarj_qLyPTkGpeoRRmNh10eC

generalif.com

Directory of Research Journals Indexing

DRJI_Logo.jpg

olddrji.lbp.world/indexedJournals.aspx

I2OR Index

8c492a0a466f9b2cd59ec89595639a5c?AccessKeyId=245B99561176BAE11FEB&disposition=0&alloworigin=1

http://www.i2or.com/8.html



logo.png