Login Register Cart 0
Generic placeholder image

Recent Advances in Computer Science and Communications

Editor-in-Chief

ISSN (Print): 2666-2558
ISSN (Online): 2666-2566

Back
Journal Home About Journal Editorial Board Current Issue Volumes/Issues
Subscribe
Review Article

Events in Tweets: Graph-Based Techniques

Author(s): Abhaya Kumar Pradhan*, Hrushikesha Mohanty and Rajendra Prasad Lal

Volume 15, Issue 2, 2022

Published on: 04 September, 2020

Page: [155 - 169] Pages: 15

DOI: 10.2174/2666255813999200904133759

Price: $65

Abstract

Background: Mining Twitter streaming posts (i.e., tweets) to find events or the topics of interest has become a hot research problem. In the last decade, researchers have come up with various techniques like bag-of-words techniques, statistical methods, graph-based techniques, topic modelling approaches, NLP and ontology-based approaches, machine learning and deep learning methods for detecting events from tweets. Among these techniques, the graph-based technique is efficient in capturing the latent structural semantics in the tweet content by modelling word cooccurrence relationships as a graph and able to capture the activity dynamics by modelling the user- tweet and user-user interactions.

Discussion: This article presents an overview of different event detection techniques and their methodologies. Specifically, this paper focuses on graph-based event detection techniques in Twitter and presents a critical survey on these techniques, their evaluation methodologies and datasets used. Further, some challenges in the area of event detection in Twitter, along with future directions of research, are presented.

Conclusion: Microblogging services and online social networking sites like Twitter provide a massive amount of valuable information on real-world happenings. There is a need for mining this information, which will help in understanding the social interest and effective decision making on various emergencies. However, event detection techniques need to be efficient in terms of time and memory and accurate for processing such voluminous, noisy and fast-arriving information from Twitter.

Keywords: Event detection, Event mining, Event identification, Topic detection, Twitter streams, Graph-based techniques, Graph mining.

Graphical Abstract

[1]
J. Allan, J.G. Carbonell, G. Doddington, J. Yamron, and Y. Yang, "Topic detection and tracking pilot study final report", In Proceedings of the DARPA Broadcast News Transcription and Understanding Workshop, 1998pp. 194-218
[2]
J. Allan, Introduction to topic detection and tracking. Inf. Retr. Ser., Springer, Boston, MA, Vol. 12, pp. 1-16, 2002.
[http://dx.doi.org/10.1007/978-1-4615-0933-2_1]
[3]
J. Sankaranarayanan, H. Samet, B.E. Teitler, M.D. Lieberman, and J. Sperling, "TwitterStand: News in tweets", In 17th ACM SIGSPATIAL International Symposium on Advances in Geographic Information Systems ACM-GIS, 2009pp. 42-51
[http://dx.doi.org/10.1145/1653771.1653781]
[4]
M. Hasan, M.A. Orgun, and R. Schwitter, TwitterNews+: A framework for real time event detection from the twitter data stream.In Lecture Notes in Computer Science, Cham:, Springer International Publishing, 2016, pp. 224-239.
[http://dx.doi.org/10.1007/978-3-319-47880-7_14]
[5]
T. Sakaki, M. Okazaki, and Y. Matsuo, "Earthquake shakes Twitter users: Real-time event detection by social sensors", Proceedings of the 19th International Conference on World Wide Web, 2010pp. 851-860
[http://dx.doi.org/10.1145/1772690.1772777]
[6]
E. Aramaki, S. Maskawa, and M. Morita, "Twitter catches the flu: Detecting influenza epidemics using Twitter", In Proceedings of the 2011 Conference on Empirical Methods in Natural Language Processing, 2011pp. 1568-1576
[7]
J. Nichols, J. Mahmud, and C. Drews, "Summarizing sporting events using Twitter", In Proceedings of the 2012 ACM International Conference on Intelligent User Interfaces, 2012pp. 189-198
[http://dx.doi.org/10.1145/2166966.2166999]
[8]
D. Corney, C. Martin, and A. Göker, "Spot the ball: Detecting sports events on Twitter", In European Conference on Information Retrieval, 2014pp. 449-454
[http://dx.doi.org/10.1007/978-3-319-06028-6_40]
[9]
P. Meladianos, G. Nikolentzos, F. Rousseau, Y. Stavrakas, and M. Vazirgiannis, "Degeneracy-based real-time sub-event detection in Twitter stream", In Proceedings of the International AAAI Conference on Web and Social Media Vol. 9, No. 1, pp. 248-257, 2015.
[10]
N.D. Prasetyo, and C. Hauff, "Twitter-based election prediction in the developing world", In Proceedings of the 26th ACM Conference on Hypertext & Social Media, 2015pp. 149-158
[http://dx.doi.org/10.1145/2700171.2791033]
[11]
W. He, S. Zha, and L. Li, "Social media competitive analysis and text mining: A case study in the pizza industry", Int. J. Inf. Manage., vol. 33, no. 3, pp. 464-472, 2013.
[http://dx.doi.org/10.1016/j.ijinfomgt.2013.01.001]
[12]
M.M. Mostafa, "More than words: Social networks’ text mining for consumer brand sentiments", Expert Syst. Appl., vol. 40, no. 10, pp. 4241-4251, 2013.
[http://dx.doi.org/10.1016/j.eswa.2013.01.019]
[13]
X. Wang, M.S. Gerber, and D.E. Brown, "Automatic crime prediction using events extracted from Twitter posts", In International Conference on Social Computing, Behavioral-Cultural Modeling and Prediction, 2012pp. 231-238
[http://dx.doi.org/10.1007/978-3-642-29047-3_28]
[14]
H. Gu, X. Xie, Q. Lv, Y. Ruan, and L. Shang, "ETree: Effective and efficient event modeling for real-time online social media networks", In 2011 IEEE/WIC/ACM International Conferences on Web Intelligence and Intelligent Agent Technology, Vol. 1, pp. 300-307, 2011.
[http://dx.doi.org/10.1109/WI-IAT.2011.126]
[15]
P. Meladianos, C. Xypolopoulos, G. Nikolentzos, and M. Vazirgiannis, "An optimization approach for sub-event detection and summarization in Twitter", In European Conference on Information Retrieval, 2018pp. 481-493
[http://dx.doi.org/10.1007/978-3-319-76941-7_36]
[16]
A. K. Pradhan, H. Mohanty, and R. P. Lal, Event detection and aspects in twitter: A bow approach. 2019, pp. 194-211.
[http://dx.doi.org/10.1007/978-3-030-05366-6_16]
[17]
J. G. Fiscus, and G. R. Doddington, Topic detection and tracking evaluation overview. In Topic Detection and Tracking, 2002, pp. 17-31.
[http://dx.doi.org/10.1007/978-1-4615-0933-2_2]
[18]
J. Li, J. Wen, Z. Tai, R. Zhang, and W. Yu, "Bursty event detection from microblog: A distributed and incremental approach", Concurrency and Computation: Practice and Experience, vol. 28, no. 11, pp. 3115-3130, 2016.
[http://dx.doi.org/10.1002/cpe.3657]
[19]
L. Yan, Handbook of research on innovative database query processing techniques., IGI Global, 2015, p. 625.
[20]
F. Kunneman, and A. Van den Bosch, "Automatically identifying periodic social events from Twitter", Proceedings of the International Conference Recent Advances in Natural Language Processing, 2015pp. 320-328
[21]
F. Atefeh, and W. Khreich, "A survey of techniques for event detection in Twitter", Comput. Intell., vol. 31, no. 1, pp. 132-164, 2015.
[http://dx.doi.org/10.1111/coin.12017]
[22]
M. Cordeiro, and J. Gama, "Online social networks event detection: A Survey, In Solving Large Scale Learning Tasks", Challenges and Algorithms, vol. 9580, pp. 1-41, 2016.
[http://dx.doi.org/10.1007/978-3-319-41706-6_1]
[23]
A. Goswami, and A. Kumar, "A survey of event detection techniques in online social networks", Soc. Netw. Anal. Min., vol. 6, no. 1, pp. 1-25, 2016.
[http://dx.doi.org/10.1007/s13278-016-0414-1]
[24]
L.M. Aiello, G. Petkos, C. Martin, D. Corney, S. Papadopoulos, R. Skraba, A. Göker, I. Kompatsiaris, and A. Jaimes, "Sensing trending topics in Twitter", IEEE Trans. Multimed., vol. 15, no. 6, pp. 1268-1282, 2013.
[http://dx.doi.org/10.1109/TMM.2013.2265080]
[25]
S. Phuvipadawat, and T. Murata, "Breaking news detection and tracking in Twitter", In 2010 IEEE/WIC/ACM International Conference on Web Intelligence and Intelligent Agent Technology, vol. Vol. 3, 2010pp. 120-123
[http://dx.doi.org/10.1109/WI-IAT.2010.205]
[26]
B. O’Connor, M. Krieger, and D. Ahn, "TweetMotif: Exploratory search and topic summarization for Twitter", In Fourth International AAAI Conference on Weblogs and Social Media, vol. Vol. 4, 2010pp. 384-385
[27]
H. Becker, M. Naaman, and L. Gravano, "Beyond trending topics: Real-world event identification on Twitter", In Proceedings of the International AAAI Conference on Web and Social Media, vol. Vol. 5, 2011pp. 438-441
[28]
S. Petrović, M. Osborne, and V. Lavrenko, "Streaming first story detection with application to Twitter", Human Language Technologies: The 2010 Annual Conference of the North American Chapter of the Association for Computational Linguistics, 2010, pp. 181- 189.
[29]
T. Brants, F. Chen, and A. Farahat, "A system for new event detection", Proceedings of the 26th Annual International ACM SIGIR Conference on Research and Development in Information Retrieval, 2003pp. 330-337
[http://dx.doi.org/10.1145/860435.860495]
[30]
W. Xie, F. Zhu, J. Jiang, E.P. Lim, and K. Wang, "TopicSketch: Real-time bursty topic detection from Twitter", IEEE Trans. Knowl. Data Eng., vol. 28, no. 8, pp. 2216-2229, 2016.
[http://dx.doi.org/10.1109/TKDE.2016.2556661]
[31]
J. Kleinberg, "Bursty and hierarchical structure in streams", Data Mining and Knowledge Discovery, vol. 7, no. 4, pp. 373-397, 2003.
[http://dx.doi.org/10.1145/775047.775061]
[32]
G.P.C. Fung, J.X. Yu, P.S. Yu, and H. Lu, "Parameter free bursty events detection in text streams", In Proceedings of the 31st International Conference on Very large Data Bases, 2005pp. 181-192
[33]
H. Sayyadi, M. Hurst, and A. Maykov, "Event detection and tracking in social streams", Third International AAAI Conference on Weblogs and Social Media, 2009pp. 311-314
[34]
M. Cataldi, L. Di Caro, and C. Schifanella, "Emerging topic detection on Twitter based on temporal and social terms evaluation", Proceedings of the Tenth International Workshop on Multimedia Data Mining, 2010pp. 1-10
[http://dx.doi.org/10.1145/1814245.1814249]
[35]
N. Alsaedi, P. Burnap, and O. Rana, "Can we predict a riot? Disruptive event detection using twitter", ACM Trans. Internet Technol., vol. 17, no. 2, pp. 1-26, 2017.
[http://dx.doi.org/10.1145/2996183]
[36]
A.K. Pradhan, and H. Mohanty, "Finding tweet events", Int. J. Comput. Appl., vol. 975, p. 8887, 2015.
[37]
D.M. Blei, A.Y. Ng, and M.I. Jordan, "Latent Dirichlet allocation", J. Mach. Learn. Res., vol. 3, pp. 993-1022, 2003.
[http://dx.doi.org/10.1016/b978-0-12-411519-4.00006-9]
[38]
L. Hong, and B.D. Davison, "Empirical study of topic modeling in Twitter", In Proceedings of the First Workshop on Social Media Analytics, 2010pp. 80-88
[http://dx.doi.org/10.1145/1964858.1964870]
[39]
C.C. Pan, and P. Mitra, "Event detection with spatial latent Dirichlet allocation", In Proceedings of the 11th Annual International ACM/IEEE Joint Conference on Digital Libraries, 2011pp. 349-358
[http://dx.doi.org/10.1145/1998076.1998141]
[40]
Y. Wang, Z. Zhang, S. Su, and M.A. Zia, "Topic-level bursty study for bursty topic detection in microblogs", In Pacific-Asia Conference on Knowledge Discovery and Data Mining, 2019pp. 97-109
[http://dx.doi.org/10.1007/978-3-030-16148-4_8]
[41]
W. Yang, D. Li, and F. Liang, "Sina Weibo bursty event detection method", IEEE Access, vol. 7, pp. 163160-163171, 2019.
[http://dx.doi.org/10.1109/ACCESS.2019.2951926]
[42]
C. Comito, A. Forestiero, and C. Pizzuti, "Bursty event detection in Twitter streams", ACM Trans. Knowl. Discov. Data, vol. 13, no. 4, pp. 1-28, 2019.
[http://dx.doi.org/10.1145/3332185]
[43]
A.M. Popescu, and M. Pennacchiotti, "Detecting controversial events from Twitter", In Proceedings of the 19th ACM International Conference on Information and Knowledge Management, 2010pp. 1873-1876
[http://dx.doi.org/10.1145/1871437.1871751]
[44]
A. Ritter, E. Wright, W. Casey, and T. Mitchell, "Weakly supervised extraction of computer security events from Twitter", In Proceedings of the 24th International Conference on World Wide Web, 2015pp. 896-905
[http://dx.doi.org/10.1145/2736277.2741083]
[45]
M. Kumar, and P. Rehan, "Graph node rank based important keyword detection from Twitter", Appl. Comput. Inform., vol. 17, no. 2, pp. 194-209, 2020.
[http://dx.doi.org/10.1016/j.aci.2018.08.002]
[46]
C. Arachie, M. Gaur, S. Anzaroot, W. Groves, K. Zhang, and A. Jaimes, "Unsupervised detection of sub-events in large scale disasters", In Proceedings of the AAAI Conference on Artificial Intelligence, vol. 34, no. 1, pp. 354-361, 2020.
[47]
U.K. Jayawardhana, and P.V. Gorsevski, "An ontology-based framework for extracting spatio-temporal influenza data using Twitter", Int. J. Digit. Earth, vol. 12, no. 1, pp. 2-4, 2019.
[http://dx.doi.org/10.1080/17538947.2017.1411535]
[48]
R. Kaushik, S.A. Chandra, D. Mallya, J.N. Chaitanya, and S.S. Kamath, "Ontology based approach for event detection in Twitter datastreams", In 2015 IEEE Region 10 Symposium, 2015, pp. 74-77.
[http://dx.doi.org/10.1109/TENSYMP.2015.19]
[49]
K. Gutiérrez-Batista, J.R. Campaña, M.A. Vila, and M.J. Martin-Bautista, "An ontology-based framework for automatic topic detection in multilingual environments", Int. J. Intell. Syst., vol. 33, no. 7, pp. 1459-1475, 2018.
[http://dx.doi.org/10.1002/int.21986]
[50]
W. Wang, Y. Ning, H. Rangwala, and N. Ramakrishnan, "A multiple instance learning framework for identifying key sentences and detecting events", In Proceedings of the 25th ACM International on Conference on Information and Knowledge Management, 2016pp. 509-518
[http://dx.doi.org/10.1145/2983323.2983821]
[51]
V.Q. Nguyen, T.N. Anh, and H.J. Yang, "Real-time event detection using recurrent neural network in social sensors", Int. J. Distrib. Sens. Netw., vol. 15, no. 6, p. 1550147719856492, 2019.
[http://dx.doi.org/10.1177/1550147719856492]
[52]
P. Goyal, P. Kaushik, P. Gupta, D. Vashisth, S. Agarwal, and N. Goyal, "Multilevel event detection, storyline generation, and summarization for Tweet streams", IEEE Transactions on Computational Social Systems, vol. 7, no. 1, pp. 8-23, 2019.
[http://dx.doi.org/10.1109/TCSS.2019.2954116]
[53]
A. Edouard, E. Cabrio, S. Tonelli, and N. Le-Thanh, Graph-based event extraction from Twitter. In RANLP17-Recent Advances in natural Language Processing, 2017, pp. 222-230.
[http://dx.doi.org/10.26615/978-954-452-049-6_031]
[54]
S.K. Biswas, M. Bordoloi, and J. Shreya, "A graph based keyword extraction model using collective node weight", Expert Syst. Appl., vol. 97, pp. 51-59, 2018.
[http://dx.doi.org/10.1016/j.eswa.2017.12.025]
[55]
M. Abulaish, S. Sharma, and M. Fazil, "A multi-attributed graph-based approach for text data modeling and event detection in Twitter", In 2019 11th International Conference on Communication Systems & Networks, 2019, pp. 703-708.
[http://dx.doi.org/10.1109/COMSNETS.2019.8711451]
[56]
N. Prangnawarat, I. Hulpuş, and C. Hayes, "Event analysis in social media using clustering of heterogeneous information networks", In 28th International Flairs Conference, 2015pp. 294-298
[57]
H. Sayyadi, and L. Raschid, "A graph analytical approach for topic detection", ACM Trans. Internet Technol., vol. 13, no. 2, pp. 1-23, 2013.
[http://dx.doi.org/10.1145/2542214.2542215]
[58]
W.D. Abilhoa, and L.N. De Castro, "A keyword extraction method from twitter messages represented as graphs", Appl. Math. Comput., vol. 240, pp. 308-325, 2014.
[http://dx.doi.org/10.1016/j.amc.2014.04.090]
[59]
T. Liu, F. Xue, J. Sun, and X. Sun, "A survey of event analysis and mining from social multimedia", Multimedia Tools Appl., vol. 79, no. 45, pp. 33431-33448, 2020.
[http://dx.doi.org/10.1007/s11042-019-7567-7]
[60]
M.A. Abebe, J. Tekli, F. Getahun, R. Chbeir, and G. Tekli, "Overview of event-based collective knowledge management in multimedia digital ecosystems", In 2017 13th International Conference on Signal-Image Technology & Internet-Based Systems, 2017, pp. 40-49.
[61]
M.A. Abebe, J. Tekli, F. Getahun, G. Tekli, and R. Chbeir, "A general multimedia representation space model toward event-based collective knowledge management", In 2016 IEEE Intl Conference on Computational Science and Engineering (CSE) and IEEE Intl Conference on Embedded and Ubiquitous Computing (EUC) and 15th Intl Symposium on Distributed Computing and Applications for Business Engineering (DCABES), 2016, pp. 512-521.
[62]
M.A. Abebe, J. Tekli, F. Getahun, R. Chbeir, and G. Tekli, "Generic metadata representation framework for social-based event detection, description, and linkage", Knowledge-Based Syst., vol. 188, p. 104817, 2020.
[http://dx.doi.org/10.1016/j.knosys.2019.06.025]
[63]
M. Zaharieva, M. Del Fabro, and M. Zeppelzauer, "Cross-platform social event detection", IEEE Multimed., vol. 22, no. 3, pp. 14-25, 2015.
[http://dx.doi.org/10.1109/MMUL.2015.31]
[64]
T. Sutanto, and R. Nayak, "Fine-grained document clustering via ranking and its application to social media analytics", Soc. Netw. Anal. Min., vol. 8, no. 1, pp. 1-9, 2018.
[http://dx.doi.org/10.1007/s13278-018-0508-z]
[65]
M. Kumar, and P. Aggarwal, "A graph based keyword extraction from twitter using node and edge weight", In 2019 International Conference on Data Science and Engineering (ICDSE), 2019pp. 35-39
[66]
Y. Sunt, J. Hant, P. Zhao, Z. Yin, H. Cheng, and T. Wu, "RankClus: Integrating clustering with ranking for heterogeneous information network analysis", In Proceedings of the 12th International Conference on Extending Database Technology: Advances in Database Technology, 2009pp. 565-576
[67]
H. Hromic, N. Prangnawarat, I. Hulpuş, M. Karnstedt, and C. Hayes, "Graph-based methods for clustering topics of interest in Twitter", In International Conference on Web Engineering, 2015pp. 701-704
[http://dx.doi.org/10.1007/978-3-319-19890-3_61]
[68]
A. Lancichinetti, F. Radicchi, J.J. Ramasco, and S. Fortunato, "Finding statistically significant communities in networks", PLoS One, vol. 6, no. 4, p. e18961, 2011.
[http://dx.doi.org/10.1371/journal.pone.0018961] [PMID: 21559480]
[69]
L. Page, and S. Brin, The anatomy of a large-scale hypertextual web search engine. Comput. Netw., Vol. 30, No. 1-7, pp. 107-117, 1998.
[http://dx.doi.org/10.1016/s0169-7552(98)00110-x]
[70]
A.J. McMinn, Y. Moshfeghi, and J.M. Jose, "Building a large-scale corpus for evaluating event detection on twitter", In Proceedings of the 22nd ACM International Conference on Information & Knowledge Management, 2013pp. 409-418
[http://dx.doi.org/10.1145/2505515.2505695]
[71]
J.H. Lau, N. Collier, and T. Baldwin, "On-line trend analysis with topic models: Twitter trends detection topic model online", In Proceedings of COLING, 2012pp. 1519-1534
[72]
B. Manaskasemsak, B. Chinthanet, and A. Rungsawang, "Graph clustering-based emerging event detection from twitter data stream", In Proceedings of the Fifth International Conference on Network, Communication and Computing, 2016pp. 37-41
[http://dx.doi.org/10.1145/3033288.3033312]
[73]
G. Salton, and C. Buckley, "Term-weighting approaches in automatic text retrieval", Inf. Process. Manage., vol. 24, no. 5, pp. 513-523, 1988.
[http://dx.doi.org/10.1016/0306-4573(88)90021-0]
[74]
S. Van Dongen, "Graph clustering via a discrete uncoupling process", SIAM J. Matrix Anal. Appl., vol. 30, no. 1, pp. 121-141, 2008.
[http://dx.doi.org/10.1137/040608635]
[75]
N. Azam, M. Abulaish, and N.A. Haldar, "Twitter data mining for events classification and analysis", In 2015 2nd International Conference on Soft Computing and Machine Intelligence (ISCMI), 2015, pp. 79-83.
[76]
X. Zhang, X. Chen, Y. Chen, S. Wang, Z. Li, and J. Xia, "Event detection and popularity prediction in microblogging", Neurocomputing, vol. 149, pp. 1469-1480, 2015.
[http://dx.doi.org/10.1016/j.neucom.2014.08.045]
[77]
S. Unankard, X. Li, and M.A. Sharaf, "Emerging event detection in social networks with location sensitivity", World Wide Web, vol. 18, no. 5, pp. 1393-1417, 2015.
[http://dx.doi.org/10.1007/s11280-014-0291-3]
[78]
I. Ruthven, and M. Lalmas, "A survey on the use of relevance feedback for information access systems", Knowl. Eng. Rev., vol. 18, no. 2, pp. 94-145, 2003.
[http://dx.doi.org/10.1017/S0269888903000638]
[79]
W. Jin, and R.K. Srihari, "Graph-based text representation and knowledge discovery", In 2007 Proceedings of the ACM Symposium on Applied Computing, 2007, pp. 807-811.
[http://dx.doi.org/10.1145/1244002.1244182]
[80]
A. Landherr, B. Friedl, and J. Heidemann, "A critical review of centrality measures in social networks", Bus. Inf. Syst. Eng., vol. 2, no. 6, pp. 371-385, 2010.
[http://dx.doi.org/10.1007/s12599-010-0127-3]
[81]
M. Garg, and M. Kumar, "TWCM: Twitter word co-occurrence model for event detection", Procedia Comput. Sci., vol. 43, pp. 434-441, 2018.
[http://dx.doi.org/10.1016/j.procs.2018.10.415]
[82]
Z. Saeed, R.A. Abbasi, M.I. Razzak, and G. Xu, "Event detection in twitter stream using weighted dynamic heartbeat graph approach", IEEE Comput. Intell. Mag., vol. 14, no. 3, pp. 29-38, 2019.
[http://dx.doi.org/10.1109/MCI.2019.2919395]
[83]
M. Fedoryszak, V. Rajaram, B. Frederick, and C. Zhong, "Real-time event detection on social data streams", Proceedings of the 25th ACM SIGKDD International Conference on Knowledge Discovery & Data Mining, 2019pp. 2774-2782
[http://dx.doi.org/10.1145/3292500.3330689]
[84]
V.D. Blondel, J.L. Guillaume, R. Lambiotte, and E. Lefebvre, "Fast unfolding of communities in large networks", J. Stat. Mech., vol. 2008, no. 10, p. 10008, 2008.
[http://dx.doi.org/10.1088/1742-5468/2008/10/P10008]
[85]
A. Bellaachia, and M. Al-Dhelaan, "NE-Rank: A novel graph-based keyphrase extraction in Twitter", In 2012 IEEE/WIC/ACM International Conferences on Web Intelligence and Intelligent Agent Technology, Vol. 1, pp. 372-379, 2012.
[http://dx.doi.org/10.1109/WI-IAT.2012.82]
[86]
J.R. Landis, and G.G. Koch, "The measurement of observer agreement for categorical data", Biometrics, vol. 33, no. 1, pp. 159-174, 1977.
[http://dx.doi.org/10.2307/2529310] [PMID: 843571]

Rights & Permissions Print Cite
Article Metrics
18
2
© 2024 Bentham Science Publishers | Privacy Policy