Academic literature on the topic 'Bibliographical citations – Evaluation'

Purpose – The academic tradition of adding a reference section with references to cited and otherwise related academic material to an article provides a natural starting point for finding links to other publications. These links can then be published as linked data. Natural language processing technologies are available today that can perform the task of bibliographical reference extraction from text. Publishing references by the means of semantic web technologies is a prerequisite for a broader study and analysis of citations and thus can help to improve academic communication in a general sense. The paper aims to discuss these issues. Design/methodology/approach – This paper examines the overall workflow required to extract, analyze and semantically publish bibliographical references within an Institutional Repository with the help of open source software components. Findings – A publication infrastructure where references are available for software agents would enable additional benefits like citation analysis, e.g. the collection of citations of a known paper and the investigation of citation sentiment.The publication of reference information as demonstrated in this article is possible with existing semantic web technologies based on established ontologies and open source software components. Research limitations/implications – Only a limited number of metadata extraction programs have been considered for performance evaluation and reference extraction was tested for journal articles only, whereas Institutional Repositories usually do contain a large number of other material like monographs. Also, citation analysis is in an experimental state and citation sentiment is currently not published at all. For future work, the problem of distributing reference information between repositories is an important problem that needs to be tackled. Originality/value – Publishing reference information as linked data are new within the academic publishing domain.

From the past half of a century, identification of the relevant documents is deemed an active area of research due to the rapid increase of data on the web. The traditional models to retrieve relevant documents are based on bibliographic information such as Bibliographic coupling, Co-citations, and Direct citations. However, in the recent past, the scientific community has started to employ textual features to improve existing models’ accuracy. In our previous study, we found that analysis of citations at a deep level (i.e., content level) can play a paramount role in finding more relevant documents than surface level (i.e., just bibliography details). We found that cited and citing papers have a high degree of relevancy when in-text citations frequency of the cited paper is more than five times in the citing paper’s text. This paper is an extension of our previous study in terms of its evaluation of a comprehensive dataset. Moreover, the study results are also compared with other state-of-the-art approaches i.e., content, metadata, and bibliography. For evaluation, a user study is conducted on selected papers from 1,200 documents (comprise about 16,000 references) of an online journal, Journal of Computer Science (J.UCS). The evaluation results indicate that in-text citation frequency has attained higher precision in finding relevant papers than other state-of-the-art techniques such as content, bibliographic coupling, and metadata-based techniques. The use of in-text citation may help in enhancing the quality of existing information systems and digital libraries. Further, more sophisticated measure may be redefined be considering the use of in-text citations.

This paper provides a discussion and analysis of methodological issues encountered during a scholarly impact and bibliometric study within the field of Computer Science (TRECVid Text Retrieval and Evaluation Conference, Video Retrieval Evaluation). The purpose of this paper is to provide a reflection and analysis of the methods used to provide useful information and guidance for those who may wish to undertake similar studies, and is of particular relevance for the academic disciplines which have publication and citation norms that may not perform well using traditional tools. Scopus and Google Scholar are discussed and a detailed comparison of the effects of different search methods and cleaning methods within and between these tools for subject and author analysis is provided. The additional database capabilities and usefulness of ‘Scopus More’ in addition to ‘Scopus General’ are discussed and evaluated. Scopus paper coverage is found to favourably compare with Google Scholar but Scholar consistently has superior performance at finding citations to those papers. These additional citations significantly increase the citation totals and also change the relative ranking of papers. Publish or Perish, a software wrapper for Google Scholar, is also examined and its limitations and some possible solutions are described. Data cleaning methods, including duplicate checks, expert domain checking of bibliographic data, and content checking of retrieved papers, are compared and their relative effects on paper and citation count discussed. Google Scholar and Scopus are also compared as tools for collecting bibliographic data for visualizations of developing trends and, owing to the comparative ease of collecting abstracts, Scopus is found far more effective.

The Covid-19 pandemic has had wide-reaching societal and economic effects and a return to “normal” will take years to accomplish. In light of this situation, the most important advancement since COVID-19′s emergence has been the development of multiple, life-saving, vaccines. Academic research on vaccine has been extensive. It is estimated that in only one year it has been produced more published and indexed papers on this single issue than in the last twenty years on any other single issue, thus, necessitating some organization. This research consists of a systematic literature review of the social science publication on COVID-19 published in the first year of the pandemic (February 2020 to March 2021). This review is important because it occurs at a time when vaccines have begun their global distribution and the best efforts to address the pandemic is through vaccination programs. In this research, 53 papers published in relevant journals are analyzed out of the almost 30,000 articles retrieved from Scopus database. The analysis conducted relies on two different types: descriptive analysis (evolution at the time of citations; evolution over time of keywords; bibliographical mapping of countries, the top 10 most influential papers), and bibliometric analysis for content evaluation. A cluster analysis was performed for the latter. Clustering the research papers, based on the actual content of papers, found there to be five research areas: (1) economic aspects; (2) ethics and legal aspects; (3) health communication; (4) policies and crisis management, and (5) political issues. Yet, this article’s results paint a picture of literature that has not yet considered the full scope of COVID-19’s effect on the economic, political, and population level health and well-being. Nor has it considered these effects across the global community, suggesting new potential areas of research and giving a perspective of what we should expect for the future.

Introduction. Citation is the main indicator in research performance evaluation using the quantitative approach. There have been many criticisms of citations since they were used half a century ago, but they have not yet succeeded in bringing new concepts and methods. This paper aims to criticize and propose a new approach to citation analysis. Data Collection Method. A contemporary critical theory methodology was adopted as a framework to collect and analyze the data. Scientific publications related to citation analysis was collected from several databases such as Google scholar, Microsoft academic search, dan Garuda Ristekdikti. Analysis Data. Publications data were critically reviewed and analyzed narratively by using open coding. Results and Discussions. The results mapped the lack of citation analysis form various aspects: (1) criticism of the positivist paradigm which did not succeed in achieving its objectives, (2) criticism of methods that produce invalid results, and (3) criticism of ethical issues of the researcher and bias in implementation. The proposed solution and recommendation is to change the citation analysis method from a simple measurement of bibliographic data to text and context analysis based on a computer science approach (machine learning techniques). Conclusion. This new method has the potential to be developed within the framework of quantitative in Science and Technology studies to overcome existing criticisms. Subsequent multidisciplinary studies are needed to lay a strong philosophical and technical foundation particularly in applying the in-text citation analysis method for evaluating research performance in accordance with the Indonesian context.

AbstractAlthough there are now several bibliographic databases of research publications, such as Google Scholar, Pubmed, Scopus, and the Web of Science (WoS), and some also include counts of citations, there is at present no similarly comprehensive database of the rapidly growing number of clinical practice guidelines (CPGs), with their references, which sometimes number in the hundreds. CPGs have been shown to be useful for the evaluation of clinical (as opposed to basic) biomedical research, which often suffers from relatively low counts of citations in the serial literature. The objectives were to introduce a new citation database, clinical impact®, and demonstrate how it can be used to evaluate research impact of clinical research publications by exploring the characteristics of CPG citations of two sets of papers, as well as show temporal variation of clinical impact® and the WoS. The paper includes the methodology used to retain the data and also the rationale adopted to achieve data quality. The analysis showed that although CPGs tend preferentially to cite papers from their own country, this is not always the case. It also showed that cited papers tend to have a more clinical research level than uncited papers. An analysis of diachronous citations in both clinical impact® and the WoS showed that although the WoS citations showed a decreasing trend after a peak at 2–3 years after publication, this was less clear for CPG citations and a longer timescale would be needed to evaluate their impact on these documents.

With the rapid development of computer science, use of artificial intelligence (AI) in education has caught much attention across the world although it is still a young field with many under-explored research elements. Through visualizing study with bibliometric evaluation and taxonomy of the literature using both VOSviewer and CiteSpace, this study provided references for readers in terms of cluster mapping on the basis of keywords, bibliographic coupling of countries, cluster mapping on the basis of co-citations, citation counts, bursts, betweenness centrality, and sigma. Researchers could also take the findings of this study into serious consideration when they set about researching effectiveness, efficiency, or usefulness of AI in education. Future research into use of AI in education will most likely need interdisciplinary cooperation between computer science, statistics, education, cognition, and robotics.

This study evaluates the content coverage of Google Scholar, as it compares to three bibliographic databases – Arts & humanities citation index, Bibliography of the history of art and Art full text/Art index retrospective – on the subject of art history. The comparison reveals that it indexes roughly one-third of the 470 tested articles, dwarfed by the Arts & humanities citation index’s 73% coverage. Additionally, 64% of the 157 articles retrieved in Google Scholar are ‘citation-only’ records that contain incomplete bibliographic data and lack direct links to the abstracts or full text of the articles.

OBJECTTVE: To compare nine on-line bibliographic databases to obtain bibliographic references on specific drug interactions. DESIGN: Seven bibliographic databases were selected for their ability to provide information concerning drug interactions: EMBASE, MEDLINE, TOXLINE, BIOSIS, Chemical Abstracts (CAS), PHARMLINE, and International Pharmaceutical Abstracts (IPA). Two French on-line bibliographic databases (i.e., PASCAL, BIBLIOGRAPHIF) were also tested to compare them with the other international databases. Twenty drug interactions were selected randomly using the journal Reactions Weekly 1993. MAIN OUTCOMES MEASURES: The total number of references, the number of potentially relevant references, the number of case report references, the number of unique references in the total number of references, and the number of unique references between potentially relevant references were analyzed by using the Friedman two-way ANOVA by ranks. For each database, relevance and relative recall were calculated. RESULTS: For the total number of references, EMBASE was significantly more comprehensive than all other databases (p < 0.05). EMBASE had a significantly greater number of potentially relevant references than IPA, PHARMLINE, CAS, and BIBLIOGRAPHIF (p < 0.05). For the total number of case report references, only one significant difference, between EMBASE and BIBLIOGRAPHIF (p < 0.05), was observed. MEDLINE and TOXLINE had the lowest cost per potentially relevant reference. CONCLUSIONS: To obtain bibliographic references on drug interactions, the first step should be to search MEDLINE or TOXLINE; the second step, for completeness, should be to search EMBASE.

The article is devoted to the debatable issues of the practice using scientific citation indices as tools of bibliographic search and citation tools. Limitations of the formal indicators’ use for bureaucratic control and evaluation of the scientific activity’s effectiveness are shown. The results of interviewing the teaching staff of the technical University in possible areas of scientific articles’ use for various communication purposes are presented.

Adult education has long been described as an emerging discipline, but there has been little empirical study of its emergence. This study examined 'emergence' by monitoring that particular knowledge base which is unique to adult education. Studies concerned with the theory and practice of adult education are a quantifiable indicator of unique knowledge about adult education. Evidence that researchers in adult education increasingly cite the work of other researchers in adult education would support the contention that the body of knowledge in adult education is growing. The articles published in Adult Education Quarterly between 1971 and 1986 were analyzed using citation analysis methodology. The frequency of citation to previous adult education studies (primary literature) as opposed to citation of studies peripheral to an adult education context (secondary literature) was determined. Distinguishing between citation categories was carried out by analyzing each title cited. The phenomenon of concern in the cited article was interpreted from the words used in the title, and coded dichotomously as 'primary literature' or 'secondary literature'. Each coded item was then recorded under named authors; thus, the cited author was credited for total frequency cited along with the coded category of writing (author of primary literature or author of secondary literature). Reliability measures performed for intra-judge consistency (recoding data), and inter-judge agreement (independent coding of data) resulted in differences in coding of less than four percent for the former, and nine percent with the latter. Validity of the procedures used in coding cited authors was tested by comparing results obtained to a 'standard'. 'Independent experts' were asked to identify from a list of the twenty most cited authors from each four volume period, those who were "primarily known for their adult education activities." The study's coding outcome of these authors compared with the expert's 'standard' resulted in greater than 75 percent agreement. With 4700 citations classified, it was found that a rising percentage of citations were to the "authors of primary literature"; from 41 percent of all citations in the first half of the study period (1971-1978), to 46 percent in the last half (1979-1986). A further breakdown showed the percentage of citations to "primary literature journals" also increasing; from 31 percent of all journals cited in '1971-1978' to 39 percent in '1979-1986'. As the scope of literature analyzed was exclusively from one North American journal, results need to be regarded with this limitation in mind. However, the empirical evidence of an increasing 'primary literature' base in adult education research suggests emergence of the field. Implications for future research are discussed in light of this and previous studies.
Education, Faculty of
Educational Studies (EDST), Department of
Graduate

近些年來，一些論文數據庫(特別是Libra) 變得公開化并已經開始提供給用戶他們請求獲得的信息。這使得我們可以將學術社區當做一個社會網絡來進行研究，不僅分析對作者的著作進行一些統計學的分析，還研究一個作者選擇性與其他作者合著的關係，以及一個作者對其他作者的影響。我們的研究即是定義一些基於社會網絡的方法來測試前面所說的影響關係及合著關係等。
我們設計的算法中，最主要的是作者影響力排名算法(AIR) 。該算法類似于著名的網頁排名算法(PageRank) ，並且把我們提取的三種關係都考慮在內。而其他的算法，都是基於某種關係或是某些關係的組合。這些算法包括:聯繫( Connection，利用合著關係、)， 追隨者數量(Follower Count ，利用發表關係)，追隨者(Follower ，利用引用關係)和平均引用數量(Balanced Citation Count ，利用合著關係和引用關係)。
這對這些算法，我們設計并研究了一些簡單的特例，通過算法之間橫向與縱向的比較來分析這些算法的特性。在不同的情形下，同一算法的表現并不一致，這是我們引入一個新的變量以便於靈活調整的原因。通過設定不同的變量值，我們利用距離衡量工具來度量這些算法結果的變化。
更進一步，我們利用不同的數據集合作為輸入來比較不同算法的表現，并利用一種距離測量工具(Spearman Footrule Distance) 來做算法之間的兩兩比較。在算法的比較中，基於排名值，我們能推斷出關於這些算法的一些結論。而基於累積值的比較，一方面驗證了這些結論的正確性，另一方面也展現出作者影響力排名算法(AIR) 的優越性。同時， 一些來源於現實生活中的排名結果，也可以用來串串證作者影響力排名算法(AIR) 的準確度。
In recent years, some of the publications database become more publically accessible, and are starting to provide additional information users can query (this is specially the case with Libra). This allows us to study the author community as a social network, analyzing not only the statistics about papers published by an author, individually at a time, but also an author’s choice and extent in connecting to other authors (co-authoring), and an author’s influence on other authors. Our approach is todesign various social network type of metrics to measure the traits defined above.
The main algorithm Author Influence Ranking (AIR), which is analogous to PageRank algorithm, is defined by taking all three relationships into consideration. Other algorithms, based on a single relationship or combination of different relationships, include: Connection, ranking algorithm using coauthor-ship; Follower Count, ranking algorithm using the number of authors who cite papers of a particular author; Follower, ranking algorithm using citation-ship; Balanced Citation Count, ranking algorithm using citation counts normalized by coauthors.
To show properties of different algorithms and do comparison among them, we design and study primitive cases. For some algorithms, the teleportation vector leads to the deviation at certain extent. Therefore, we study different teleportation vectors by tuning a parameter β(details discussed in Chapter 4) from 0 (uniformly distributed) to 1 (proportional to authors’ productivity). With different β, we define Distance to measure the changes in results of these algorithms.
Furthermore, comparisons among these algorithms are conducted by using different publication dataset and we choose Spearman Footrule Distance in our experiment to do comparison for pair of algorithms. Rank value and cumulative value are used in the comparisons: based on the comparisons using rank value, we conclude several observations regarding these algorithms. While the comparisons based on cumulative value help us confirm the "efficiency" of AIR. For using AIR metric, we can find out those really influential researchers who may not be ranked high by other metrics. We study the influence of Turing award winners and all the Turing Award winners scored at least "B", from which we can see AIR’s "accuracy". We also apply AIR metric in the real situation. We study researchers who have Grade "A"(the grade will be discussed in Chapter 6) in Influence and find most of them have good positions in reality, which help us justify the validity of AIR.("efficiency", "accuracy" and "validity" will be discussed more in Chapter 6.)
Detailed summary in vernacular field only.
Detailed summary in vernacular field only.
Detailed summary in vernacular field only.
Detailed summary in vernacular field only.
Song, Qianqian.
Thesis (M.Phil.)--Chinese University of Hong Kong, 2013.
Includes bibliographical references (leaves 78-82).
Abstracts also in Chinese.
Abstract --- p.i
Acknowledgement --- p.iv
Chapter 1 --- Introduction --- p.1
Chapter 1.1 --- Motivation --- p.1
Chapter 1.2 --- Background --- p.2
Chapter 1.3 --- Our Contribution and Organization --- p.5
Chapter 2 --- Academic Ranking Algorithms --- p.8
Chapter 2.1 --- Publication Statistics Algorithms --- p.10
Chapter 2.1.1 --- Citation Count --- p.10
Chapter 2.1.2 --- Balanced Citation Count --- p.11
Chapter 2.1.3 --- Follower Count --- p.12
Chapter 2.2 --- PageRank-like Algorithms --- p.13
Chapter 2.2.1 --- The PageRank Algorithm --- p.13
Chapter 2.2.2 --- Author Influence Ranking --- p.15
Chapter 2.2.3 --- Science Author Rank Algorithm --- p.17
Chapter 2.2.4 --- Connection --- p.20
Chapter 2.2.5 --- Follower --- p.21
Chapter 3 --- Analysis of Metrics Based on Primitive Cases --- p.22
Chapter 3.1 --- The Original Case --- p.23
Chapter 3.2 --- Case for Three General Authors --- p.24
Chapter 3.3 --- Case for Productive Authors --- p.26
Chapter 3.4 --- Cases for Productive Author and Coauthor-ship --- p.28
Chapter 3.4.1 --- Type i --- p.28
Chapter 3.4.2 --- Type ii --- p.30
Chapter 3.5 --- Case for Coauthor-ship --- p.32
Chapter 3.6 --- Cases for Citation Count and Balanced Citation Count --- p.34
Chapter 3.6.1 --- Type i --- p.34
Chapter 3.6.2 --- Type ii --- p.36
Chapter 4 --- Key Parameter in PageRank-like Algorithms --- p.39
Chapter 4.1 --- The Key Parameter β --- p.39
Chapter 4.2 --- Comparison Based on β --- p.40
Chapter 4.3 --- Discussion --- p.43
Chapter 5 --- Algorithms Comparison --- p.46
Chapter 5.1 --- The Description of Our Comparisons --- p.46
Chapter 5.2 --- Similarity Between Different Metrics --- p.47
Chapter 5.3 --- Two Dimensions Comparison --- p.51
Chapter 5.3.1 --- Comparison in Algorithms Dimension --- p.51
Chapter 5.3.2 --- Comparison in Time Dimension --- p.54
Chapter 6 --- Case Study and Validation --- p.56
Chapter 6.1 --- AIR v.s Other Metrics --- p.57
Chapter 6.1.1 --- AIR v.s Citation Count --- p.58
Chapter 6.1.2 --- AIR v.s Follower Count --- p.59
Chapter 6.1.3 --- AIR v.s Follower --- p.61
Chapter 6.1.4 --- AIR v.s Connection --- p.62
Chapter 6.1.5 --- AIR v.s the First Active Year --- p.64
Chapter 6.2 --- AIR v.s Rank in Reality --- p.65
Chapter 6.2.1 --- Ranking Award Recipients --- p.65
Chapter 6.2.2 --- Top AIR Ranking in Society --- p.65
Chapter 7 --- Conclusion --- p.76
Bibliography --- p.78