A cost effective approach to establish benchmark for automatic workpiece classification systems

Shang Hwa Hsu*, Tai Chang Hsia, Muh-Cherng Wu

*Corresponding author for this work

Research output: Contribution to journalArticlepeer-review


The utility of an automatic workpiece classification system depends primarily on the extent to which its classification results are consistent with users' judgments. Thus to evaluate the effectiveness of an automatic classification system it is necessary to establish classification benchmarks based on users' judgments. Such benchmarks are typically established by having subjects perform pair comparisons of all workpieces in a set of sample workpieces. The result of such comparisons is called a full-data classification. However, when the number of sample workpieces is very large, such exhaustive comparisons become impractical. This paper proposes a more efficient method, called lean-data classification, in which data on some pair comparison are used to infer the complete pair comparison results. The proposed method has been verified by using a set of 36 sample workpieces. The result revealed that the method could produce a classification that was 78% consistent with the full-data classification while using only 40% of the total data.

Original languageEnglish
Pages (from-to)112-122
Number of pages11
JournalInternational Journal of Industrial Engineering : Theory Applications and Practice
Issue number2
StatePublished - 1 Jun 2002


  • Automatic workpiece classification system
  • Classification benchmarks
  • Full-data classification
  • Lean-data classification


Dive into the research topics of 'A cost effective approach to establish benchmark for automatic workpiece classification systems'. Together they form a unique fingerprint.

Cite this