Login Register Cart 0
Generic placeholder image

Recent Advances in Computer Science and Communications

Editor-in-Chief

ISSN (Print): 2666-2558
ISSN (Online): 2666-2566

Back
Journal Home About Journal Editorial Board Current Issue Volumes/Issues
Subscribe
Research Article

Comparative Performance Evaluation of Keyword and Semantic Search Engines using Different Query Set Categories

Author(s): Poonam Jatwani*, Pradeep Tomar and Vandana Dhingra

Volume 13, Issue 5, 2020

Page: [1057 - 1070] Pages: 14

DOI: 10.2174/2213275912666190328202153

Price: $65

Abstract

Background: Keyword search engines are unable to understand the intention of user as a result they produce enormous results for user to distinguish between relevant and non relevant answers of user queries. This has led to rise in requirement to study search capabilities of different search engines. In this research work, experimental evaluation is done based on different metrics to distinguish different search engines on the basis of type of query that can be handled by them.

Methods: To check the semantics handling performance, four types of query sets consisting of 20 queries of agriculture domain are chosen. Different query set are single term queries, two term queries, three term queries and NLP queries. Queries from different query set were submitted to Google, DuckDuckGo and Bing search engines. Effectiveness of different search engines for different nature of queries is experimented and evaluated in this research using Grade relevance measures like Cumulative Gain, Discounted Cumulative Gain, Ideal Discounted Cumulative Gain, and Normalized Discounted Cumulative Gain in addition to the precision metric.

Results: Our experimental results demonstrate that for single term query, Google retrieves more relevant documents and performs better and DuckDuckGo retrieves more relevant documents for NLP queries.

Conclusion: Analysis done in this research shows that DuckDuckGo understand human intention and retrieve more relevant result, through NLP queries as compared to other search engines.

Keywords: Precision, semantic web, search engine evaluation, natural language processing (NLP), query, DuckDuckGo.

Graphical Abstract

[1]
S. Brin, and L. Page, "The anatomy of large-scale hyper textual web search engine", Computer Networks and ISDN System, vol. 30, no. 1-7. 1998, pp. 107-117.
[2]
S. Gupta, and D. Garg, "Comparison of semantic and syntactic information retrieval system on the basis of precision and recall", Int. J. Data Engineering (IJDE), vol. 2, no. 3, pp. 93-101, 2011.
[3]
T. Berners-Lee, J. Hendler, and O. Lassila, "The Semantic web", Sci. Am., vol. 284, no. 5, pp. 34-43, 2001.
[4]
X. Shen, Y. Xu, J. Yu, and K. Zhang, "Intelligent search engine based on formal concept analysis In", IEEE International Conference on Granular Computing, 2007pp. 669-669
[5]
D. Ding, J. Yang, Q. Li, L. Wang, and W. Liu, "Towards a flash search engine based on expressive semantics In", Proceedings of International World Wide Web Conference on Alternate Track, WWW Alt, New York, 2004pp. 472-473
[6]
K. Dahlgren, and D. Albro, "Cognition Technology resources overview semantic Map system architecture and tools at cognition giving new meaning", , 2008.
[7]
T.R. Gruber, "A translation approach to portable ontology Specification", Knowl. Acquis., pp. 199-220, 1993.
[8]
S.S. Laddha, and P.M. Jawandhiya, "Semantic search engine", Indian J. Sci. Technol., vol. 10, no. 23, pp. 1-6, 2017.
[9]
G. Madhu, A. Govardhan, and T.V. Rajinikanth, "Intelligent semantic web search engines: a brief survey", Int. J. Web & Semantic Tech. (IJWesT), vol. 2, no. 1, pp. 34-42, 2011.
[10]
D. Hawking, N. Craswell, P. Bailey, and K. Griffiths, "Measuring search engine quality", Inf. Retrieval, vol. 4, pp. 33-59, 2001.
[11]
M. Tang, and Y. Sun, "“Evaluation of web-based search enginesusing user effort measures”", Lib. and Inform. Sci. Res. Electronic J.(LIBRES), vol. 3, no. 2, 2003.
[12]
S. M. Shafi, and R. A. Rather, "Precision and recall of five search engines for retrieval of scholarly information in the field of biotechnology", In: Webology, vol. 2, no. 2. 2005.
[13]
H. Girit, R. Eberhard, B. Michelberger, and B. Mutschler, "On the precision of search engines: Results from a controlled experiment", In: W. Abramowicz, D. Kriksciuniene, and V. Sakalauskas, Eds., Business Information Systems. BIS 2012. Lecture Notes in Business Information Processing., vol. 117. Springer: Berlin, Heidelberg, 2005.
[14]
R.A. Rather, F.A. Lone, and G.J. Shah, Overlap in web search results: a study of five search engines., Library Philosophy and Practice, 2008.
[15]
D. Tumer, M.A. Shah, and Y. Bitirim, "An empirical evaluation on semantic search performance of keyword-based and semantic search engines: Google, Yahoo, Msn and Hakia In", Fourth International Conference on Internet Monitoring and Protection, 2009pp. 51-55
[16]
B.T.S. Kumar, and S.M. Pavithra, Evaluating the searching capabilities of search engines and metasearch engines: A comparative study., Ann. Lib. Inform. Studies, pp. 87-97, 2010.
[17]
M.O. Andago, T.P.L. Phoebe, and B.A.M. Thanoun, "Evaluation of a semantic search engine against a keyword search engine using first 20 precision", Proceedings of the 29th Annual International Conference on Research and Development in Information Retrieval, ACM Press, pp. 735-746, 2010.
[18]
M.M. Qureshi, B. Asma, and H.U. Khan, "Comparative analysis of semantic search engines based on requirement space pyramid", Int. J. Future Comp. Comm., vol. 2, no. 6, pp. 562-566, 2013.
[19]
Y.S. Negi, and S. Kumar, A comparative analysis of keyword and semantic based search engines. InIn Intelligent Computing, Networking, and Informatics,, 2014, pp. 727-736.
[20]
J.A. Khan, D. Sangroha, M. Ahmad, and M.T. Rahman, "A performance evaluation of semantic-based search engines and keyword-based search engines", In Medical imaging, m-health andemerging communication systems (MedCom), IEEE International Conference, 2014pp. 168-173
[21]
L. Ding, T. Finin, A. Joshi, Y. Peng, R.S. Cost, J. Sachs, R. Pan, P. Reddivari, V. Doshi, and P. Reddivari, "Swoogle: A search and metadata engine for the semantic web", Proceedings of ACM Conference on Information and Knowledge Management (CIKM’04), 2004pp. 652-659 Washington DC, USA
[22]
C. Mangold, "A survey and classification of semantic search approaches", Int. J. Metadata Semant. Ontol., vol. 2, no. 1, pp. 22-34, 2007.
[23]
Y. Qu, G. Cheng, H. Wu, W. Ge, and X. Zhang, Seeking Knowledge with Falcons., Semantic Web Challenges, 2008.
[24]
P. J. Hane, "Cognition giving new meaning", J. Nat. Lang. Processing, pp. 90-99, 2010.
[25]
S. Mouhim, A. El aoufi, A. EI Mezouary, M. Edahibi, H. Eddouzi, C. Cherkaoui, and D. Mammas, "“A practical and functional evolution of some semantic search engines”", Int. J. Comp. Sci. and Inform. Tech. Security (IJCSITS), vol. 2, no. 5, 2012.
[26]
K. Jarvelin, and J. Kekalainen, "Cumulated gain based evaluation of IR Techniques", ACM Trans. Inf. Syst., vol. 20, no. 4, pp. 422-446, 2002.
[27]
D. Yin, C. Nobata, and J.M. Langlois, Yichang et al"Ranking relevance in yahoo search", Proceeding of the 22nd ACM SIGKDD International Conference on Knowledge Discovery and Data Mining, 2016pp. 323-332
[28]
Y. Wang, L. Wang, Y. Li, D. He, W. Chen, and T.Y. Liu, "A theoretical analysis of NDCG ranking measures In", Proceedings of the 26th annual conference on Learning Theory, 2013
[29]
B. Chandrasekaran, J.R. Josephson, and V.R. Benjamins, "What are ontologies, and why do we need them?", IEEE Intell. Syst., vol. 14, no. 1, pp. 20-26, 1999.

Rights & Permissions Print Cite
Article Metrics
24
© 2024 Bentham Science Publishers | Privacy Policy