The rapid expansion of scientific literature has made it challenging for
researchers to find relevant studies on specific topics efficiently. The traditional search
methods require extensive time and manual effort to identify, filter, and extract
essential information from numerous sources. This paper proposes an automated
keyword-based web crawling system aimed at streamlining the retrieval of research
papers and patents from sources, IEEE and Google patents, respectively. In the
proposed work, the authors type the keyword of the research paper or patent, and then
the system provides the results accordingly. The details of a patent or research paper
may include details like the name of the author, the DOI, the publisher, the title of the
article, publication date, and many more. The proposed technique decreases manual
work and enhances the accuracy of collecting the required data.
Keywords: Personalized web crawler, Patent scraping, Patent data mining, Selenium automation.
