Crowd-Sourced Knowledge Bases

Yang Sok Kim, Byeong Ho Kang, Seung Hwan Ryu, Paul Compton, Soyeon Caren Han, Tim Menzies

Research output: Chapter in Book/Conference paperConference paperpeer-review


Crowdsourcing is a low cost way of obtaining human judgements on a large number of items, but the knowledge in these judgements is not reusable and further items to be processed require further human judgement. Ideally one could also obtain the reasons people have for these judgements, so the ability to make the same judgements could be incorporated into a crowd-sourced knowledge base. This paper reports on experiments with 27 students building knowledge bases to classify the same set of 1000 documents. We have assessed the performance of the students building the knowledge bases using the same students to assess the performance of each other’s knowledge bases on a set of test documents. We have explored simple techniques for combining the knowledge from the students. These results suggest that although people vary in document classification, simple merging may produce reasonable consensus knowledge bases.

Original languageEnglish
Title of host publicationKnowledge Management and Acquisition for Intelligent Systems - 12th Pacific Rim Knowledge Acquisition Workshop, PKAW 2012, Proceedings
EditorsDeborah Richards, Byeong Ho Kang
PublisherSpringer-Verlag Italia Srl
Number of pages14
ISBN (Print)9783642325403
Publication statusPublished - 2012
Externally publishedYes
EventInternational Workshop on Knowledge Management and Acquisition for Intelligent Systems 2012 - Kuching, Malaysia
Duration: 5 Sept 20126 Sept 2012
Conference number: 12

Publication series

NameLecture Notes in Computer Science (including subseries Lecture Notes in Artificial Intelligence and Lecture Notes in Bioinformatics)
Volume7457 LNAI
ISSN (Print)0302-9743
ISSN (Electronic)1611-3349


ConferenceInternational Workshop on Knowledge Management and Acquisition for Intelligent Systems 2012
Abbreviated titlePKAW 2012


Dive into the research topics of 'Crowd-Sourced Knowledge Bases'. Together they form a unique fingerprint.

Cite this