先来一个简单的测试

# curl -XPOST  "http://192.168.9.155:9200/_analyze?analyzer=standard&pretty" -d 'PHP是世界上最好的语言'   //_analyze表示分析分词;analyzer=standard,表示分词方式standard; -d表示测试的一段文字

测试结果

{ 
  "tokens" : [ 
    { 
      "token" : "php", 
      "start_offset" : 0, 
      "end_offset" : 3, 
      "type" : "<ALPHANUM>", 
      "position" : 0 
    }, 
    { 
      "token" : "", 
      "start_offset" : 3, 
      "end_offset" : 4, 
      "type" : "<IDEOGRAPHIC>", 
      "position" : 1 
    }, 
    { 
      "token" : "", 
      "start_offset" : 4, 
      "end_offset" : 5, 
      "type" : "<IDEOGRAPHIC>", 
      "position" : 2 
    }, 
    { 
      "token" : "", 
      "start_offset" : 5, 
      "end_offset" : 6, 
      "type" : "<IDEOGRAPHIC>", 
      "position" : 3 
    }, 
    { 
      "token" : "", 
      "start_offset" : 6, 
      "end_offset" : 7, 
      "type" : "<IDEOGRAPHIC>", 
      "position" : 4 
    }, 
    { 
      "token" : "", 
      "start_offset" : 7, 
      "end_offset" : 8, 
      "type" : "<IDEOGRAPHIC>", 
      "position" : 5 
    }, 
    { 
      "token" : "", 
      "start_offset" : 8, 
      "end_offset" : 9, 
      "type" : "<IDEOGRAPHIC>", 
      "position" : 6 
    }, 
    { 
      "token" : "", 
      "start_offset" : 9, 
      "end_offset" : 10, 
      "type" : "<IDEOGRAPHIC>", 
      "position" : 7 
    }, 
    { 
      "token" : "", 
      "start_offset" : 10, 
      "end_offset" : 11, 
      "type" : "<IDEOGRAPHIC>", 
      "position" : 8 
    }, 
    { 
      "token" : "", 
      "start_offset" : 11, 
      "end_offset" : 12, 
      "type" : "<IDEOGRAPHIC>", 
      "position" : 9 
    } 
  ] 
}

接下来使用我们的IK

ik 带有两个分词器 
ik_max_word :会将文本做最细粒度的拆分;尽可能多的拆分出词语,拼接各种可能的组合 。
ik_smart:会做最粗粒度的拆分;已被分出的词语将不会再次被其它词语占有 。

curl -XPOST  "http://192.168.9.155:9200/_analyze?analyzer=ik_smart&pretty" -d 'PHP是世界上最好的语言'  //ik_smart方式
{ 
  "tokens" : [ 
    { 
      "token" : "php", 
      "start_offset" : 0, 
      "end_offset" : 3, 
      "type" : "ENGLISH", 
      "position" : 0 
    }, 
    { 
      "token" : "世界上", 
      "start_offset" : 4, 
      "end_offset" : 7, 
      "type" : "CN_WORD", 
      "position" : 1 
    }, 
    { 
      "token" : "最好", 
      "start_offset" : 7, 
      "end_offset" : 9, 
      "type" : "CN_WORD", 
      "position" : 2 
    }, 
    { 
      "token" : "语言", 
      "start_offset" : 10, 
      "end_offset" : 12, 
      "type" : "CN_WORD", 
      "position" : 3 
    } 
  ] 
}
curl -XPOST  "http://192.168.9.155:9200/_analyze?analyzer=ik_max_word&pretty" -d 'PHP是世界上最好的语言'    //ik_max_word方式
{ 
  "tokens" : [ 
    { 
      "token" : "php", 
      "start_offset" : 0, 
      "end_offset" : 3, 
      "type" : "ENGLISH", 
      "position" : 0 
    }, 
    { 
      "token" : "世界上", 
      "start_offset" : 4, 
      "end_offset" : 7, 
      "type" : "CN_WORD", 
      "position" : 1 
    }, 
    { 
      "token" : "世界", 
      "start_offset" : 4, 
      "end_offset" : 6, 
      "type" : "CN_WORD", 
      "position" : 2 
    }, 
    { 
      "token" : "", 
      "start_offset" : 6, 
      "end_offset" : 7, 
      "type" : "CN_CHAR", 
      "position" : 3 
    }, 
    { 
      "token" : "最好", 
      "start_offset" : 7, 
      "end_offset" : 9, 
      "type" : "CN_WORD", 
      "position" : 4 
    }, 
    { 
      "token" : "语言", 
      "start_offset" : 10, 
      "end_offset" : 12, 
      "type" : "CN_WORD", 
      "position" : 5 
    } 
  ] 
}

区别很明显~

发布评论

分享到:

IT虾米网

微信公众号号:IT虾米 (左侧二维码扫一扫)欢迎添加!

ElasticSearch(五) Elasticsearch-jdbc实现MySQL同步到ElasticSearch详解
你是第一个吃螃蟹的人
发表评论

◎欢迎参与讨论,请在这里发表您的看法、交流您的观点。