ElasticSearch之cardinality，range，extended_stats聚合统计aggregations查询

kuangbin 2022年11月07日编程语言 134 0

ElasticSearch查询笔记目录

涉及的常用查询内容较多，将分多个章节进行笔记整理，具体如下：

ElasticSearch查询学习笔记章节1——term，terms，match，id查询

主要是依据精准的查询条件来查询，查询速度快，也是最常用的几类查询方式，具体种类如下:

term查询
terms查询
match_all查询
match查询
布尔match查询
multi_match查询
根据文档id查询（单个id）
根据文档ids查询（多个id）

ElasticSearch查询学习笔记章节2——prefix，fuzzy，wildcard，range，regexp查询

主要是涉及ElasticSearch查询条件相对模糊，查询速度相对慢，实时查询时应尽量避免这些方式，但是这些查询方式又具有自己独特不可代替的功能，还是还有必要，具体如下：

prefix查询
fuzzy查询
wildcard查询
range查询
regexp查询

ElasticSearch查询学习笔记章节3——scroll，delete-by-query，bool，boosting，filter，highlight查询

主要涉及ElasticSearch的一些常用的杂项查询；

深分页scroll查询
delete-by-query
bool查询
boosting查询
filter查询
highlight(高亮)查询

ElasticSearch查询学习笔记章节4——cardinality，range，extended_stats聚合统计aggregations查询

主要涉及ES的聚合查询Aggregations；

cardinality(去重计数)查询
range(范围统计)查询
extended_stats(统计聚合)查询

ElasticSearch查询学习笔记章节5——geo_distance，geo_bounding_box，geo_polygon地图检索geo查询

. 主要涉及ES的地图检索geo相关的查询；

geo_distance查询
geo_bounding_box查询
geo_polygon查询

整体Java代码的测试用例项目

整个章节的Java代码放在CSDN资源ElasticSearch常用查询的Java实现；路径效果如下图，欢迎下载访问；

前言

该章节笔记主要针对非常重要的聚合查询，具体如下，ES的聚合查询和MySQL的聚合查询类型类似，但是ES的聚合查询相比于MySQL要强大的多，ES提供的统计数据的方式多种多样，这里就挑选几个常用的来示范一下，更多的内容可以参考官网Aggregations;
ES聚合查询的RESTFul代码语法如下；

POST /index/_search   #这里和普通查询一样，一般用POST或者GET，后面接索引名和_search,用/隔开 
{
    
	"aggs":{
          #这里和普通的query不一样了，不是写query，是写aggs，是aggregations的缩写                
	    "名字，一般用agg":  #这里指定此次聚合计算的一个名字，一般都起名agg 
	    {
    
	        "agg_type":  #选择系统内的聚合函数类型 
	        {
    
	            "属性":"值"   #指定需要聚合计算的属性和值 
	        } 
	    } 
	} 
}

cardinality(去重计数)查询

去重计算，即cardinality,类似于MySQL的count(distinct field)，第一步先将返回的文档中的一个指定的filed进行去重，第二步，再统计一下去重后总共有多少条数据；

实现要求，依据province字段查询所有公司驻场在几个城市，即去重province后，最终有几个城市。

RESTFUL代码如下；

#cardinality查询 
POST /sms-logs-index/_search 
{
    
  "aggs": {
    
    "agg": {
    
      "cardinality": {
    
        "field": "province" 
      } 
    } 
  } 
}

注意：cardinality的优点性能快，亿级别的记录在1秒内完成，但是存在只能保证最大40000条记录内的精确，超过的存在5%的误差，不适合需要精确去重场景！参数precision_threshold接受 0–40000 之间的数字，更大的值还是会被当作 40000 来处理。precision_threshold值设置的越大，占用内存越大，样例如下；

GET /myindex/mytype/_search 
{
    
    "size" : 0, 
    "aggs" : {
    
        "distinct_idCard" : {
    
            "cardinality" : {
    
              "field" : "idCard", 
              "precision_threshold" : 100  
            } 
        } 
    } 
}

RESTFUL代码的查询结果如下，注意看，结果是在最后的跟第二个hits平级的标签aggregations标签里面，里面的agg是我们取的此次的聚合查询的名字，名字发生变化的话这个值也会发生变化，value对应的值3,代表着最后的返回结果；

{
    
  "took" : 14, 
  "timed_out" : false, 
  "_shards" : {
    
    "total" : 5, 
    "successful" : 5, 
    "skipped" : 0, 
    "failed" : 0 
  }, 
  "hits" : {
    
    "total" : {
    
      "value" : 12, 
      "relation" : "eq" 
    }, 
    "max_score" : 1.0, 
    "hits" : [ 
      {
    
        "_index" : "sms-logs-index", 
        "_type" : "_doc", 
        "_id" : "3", 
        "_score" : 1.0, 
        "_source" : {
    
          "createDate" : "2020-09-16", 
          "senDate" : "2020-09-16", 
          "longCode" : "5478434123", 
          "moblie" : 18056587445, 
          "corpName" : "中威集团", 
          "smsContent" : "中威集团，服务于你的身边！", 
          "state" : "0", 
          "opratorId" : "3", 
          "province" : "杭州", 
          "ipAddr" : "10.248.19.45", 
          "replyTotal" : "4", 
          "fee" : "20" 
        } 
      }, 
      {
    
        "_index" : "sms-logs-index", 
        "_type" : "_doc", 
        "_id" : "5", 
        "_score" : 1.0, 
        "_source" : {
    
          "createDate" : "2020-09-16", 
          "senDate" : "2020-09-16", 
          "longCode" : "24514635", 
          "moblie" : 18545427895, 
          "corpName" : "东东集团", 
          "smsContent" : "数据驱动，AI推动，新零售模型让你的购买更心怡！", 
          "state" : "1", 
          "opratorId" : "1", 
          "province" : "北京", 
          "ipAddr" : "10.254.19.45", 
          "replyTotal" : "1", 
          "fee" : "6000" 
        } 
      }, 
      {
    
        "_index" : "sms-logs-index", 
        "_type" : "_doc", 
        "_id" : "11", 
        "_score" : 1.0, 
        "_source" : {
    
          "createDate" : "2020-09-22", 
          "senDate" : "2020-09-22", 
          "longCode" : "458744536", 
          "moblie" : 134625584654, 
          "corpName" : "星雨文化传媒", 
          "smsContent" : "魅力宣传，星雨传媒！", 
          "state" : "1", 
          "opratorId" : "3", 
          "province" : "杭州", 
          "ipAddr" : "10.289.19.45", 
          "replyTotal" : "6", 
          "fee" : "500" 
        } 
      }, 
      {
    
        "_index" : "sms-logs-index", 
        "_type" : "_doc", 
        "_id" : "9", 
        "_score" : 1.0, 
        "_source" : {
    
          "createDate" : "2020-09-16", 
          "senDate" : "2020-09-16", 
          "longCode" : "5784320", 
          "moblie" : 15236964578, 
          "corpName" : "花花派", 
          "smsContent" : "花开花落，魅力女性，买花选我！", 
          "state" : "1", 
          "opratorId" : "1", 
          "province" : "上海", 
          "ipAddr" : "10.265.19.45", 
          "replyTotal" : "1", 
          "fee" : "0.1" 
        } 
      }, 
      {
    
        "_index" : "sms-logs-index", 
        "_type" : "_doc", 
        "_id" : "4", 
        "_score" : 1.0, 
        "_source" : {
    
          "createDate" : "2020-09-16", 
          "senDate" : "2020-09-16", 
          "longCode" : "87454120", 
          "moblie" : 13625789645, 
          "corpName" : "爱美化妆品有限公司", 
          "smsContent" : "魅力，势不可挡，爱美爱美", 
          "state" : "1", 
          "opratorId" : "1", 
          "province" : "上海", 
          "ipAddr" : "10.258.19.45", 
          "replyTotal" : "1", 
          "fee" : "200" 
        } 
      }, 
      {
    
        "_index" : "sms-logs-index", 
        "_type" : "_doc", 
        "_id" : "7", 
        "_score" : 1.0, 
        "_source" : {
    
          "createDate" : "2020-09-16", 
          "senDate" : "2020-09-16", 
          "longCode" : "33656412674", 
          "moblie" : 18956451203, 
          "corpName" : "华丽网集团", 
          "smsContent" : "网络安全，华丽靠谱！", 
          "state" : "1", 
          "opratorId" : "3", 
          "province" : "上海", 
          "ipAddr" : "10.215.254.45", 
          "replyTotal" : "1", 
          "fee" : "2000" 
        } 
      }, 
      {
    
        "_index" : "sms-logs-index", 
        "_type" : "_doc", 
        "_id" : "8", 
        "_score" : 1.0, 
        "_source" : {
    
          "createDate" : "2020-09-16", 
          "senDate" : "2020-09-16", 
          "longCode" : "56412345", 
          "moblie" : 17055452369, 
          "corpName" : "万事Ok公司", 
          "smsContent" : "万事Ok，找我没错！", 
          "state" : "0", 
          "opratorId" : "2", 
          "province" : "杭州", 
          "ipAddr" : "10.215.19.45", 
          "replyTotal" : "1", 
          "fee" : "200" 
        } 
      }, 
      {
    
        "_index" : "sms-logs-index", 
        "_type" : "_doc", 
        "_id" : "10", 
        "_score" : 1.0, 
        "_source" : {
    
          "createDate" : "2020-09-16", 
          "senDate" : "2020-09-16", 
          "longCode" : "54784641", 
          "moblie" : 15625584654, 
          "corpName" : "勾股科技有限公司", 
          "smsContent" : "智能算法，智慧生活，勾股科技！", 
          "state" : "1", 
          "opratorId" : "2", 
          "province" : "杭州", 
          "ipAddr" : "10.215.19.45", 
          "replyTotal" : "6", 
          "fee" : "4000" 
        } 
      }, 
      {
    
        "_index" : "sms-logs-index", 
        "_type" : "_doc", 
        "_id" : "2", 
        "_score" : 1.0, 
        "_source" : {
    
          "createDate" : "2020-09-16", 
          "senDate" : "2020-09-16", 
          "longCode" : "20165411010", 
          "moblie" : 15248754897, 
          "corpName" : "北京鑫鑫能源有限公司", 
          "smsContent" : "欢迎使用新能源，让世界更环保", 
          "state" : "1", 
          "opratorId" : "2", 
          "province" : "北京", 
          "ipAddr" : "10.245.29.280", 
          "replyTotal" : "0.6", 
          "fee" : "0.5" 
        } 
      }, 
      {
    
        "_index" : "sms-logs-index", 
        "_type" : "_doc", 
        "_id" : "6", 
        "_score" : 1.0, 
        "_source" : {
    
          "createDate" : "2020-09-16", 
          "senDate" : "2020-09-16", 
          "longCode" : "89451254", 
          "moblie" : 13028457893, 
          "corpName" : "大兴建筑有限公司", 
          "smsContent" : "我房建，你放心，大兴建筑！", 
          "state" : "1", 
          "opratorId" : "1", 
          "province" : "杭州", 
          "ipAddr" : "10.215.19.45", 
          "replyTotal" : "1", 
          "fee" : "500" 
        } 
      } 
    ] 
  }, 
  "aggregations" : {
    
    "agg" : {
    
      "value" : 3 
    } 
  } 
}

Java代码如下；

    static RestHighLevelClient myClient= EsClient.getClient();  //获取操作ES的 
    String index="sms-logs-index"; 
 
    @Test 
    public void cardnality() throws IOException 
    {
    
        //1.SearchRequest 
        SearchRequest request=new SearchRequest(index); 
 
        //2.聚合查询条件 
        SearchSourceBuilder builder=new SearchSourceBuilder(); 
        builder.aggregation(AggregationBuilders.cardinality("agg").field("province")); 
        request.source(builder); 
 
        //3.执行查询 
        SearchResponse resp = myClient.search(request, RequestOptions.DEFAULT); 
 
        //4.返回结果，默认返回的是父类Aggregation 
        Aggregation agg = resp.getAggregations().get("agg"); 
 
        //转换成子接口Cardinality才有getValue()方法，父接口Aggregation没有该方法 
        Cardinality myagg=(Cardinality) agg; 
         
        //以上两句代码也可以写成直接写成    Cardinality agg = resp.getAggregations().get("agg"); 
        //后续所有的聚合查询都要有此类似的操作，切记！ 
 
        long value = myagg.getValue(); 
        System.out.println(value); 
 
 
    }

Java代码实现的效果如图1如下；

图1 Java实现cardinality的效果

range(范围统计)查询

统计一定范围内出现的文档个数，比如。针对某一个field的值在0到100,100到200,200到300之间的文档出现的个数分别是多少。
范围统计可以针对普通的数值，针对时间，针对ip类型都可以做相应的统计。
range，date_range,ip_range
range统计这里涉及参数：
from：大于等于
to：小于

数值range统计：实现要求，依据fee字段查询费用小于200的；大于等于200，小于400；大于等于400的分别有多少公司

RESTFUL代码如下；

POST /sms-logs-index/_search 
{
    
  "aggs": {
    
    "agg": {
    
      "range": {
    
        "field": "fee", 
        "ranges": [ 
          {
    
            "to": 200 
          }, 
          {
    
            "from": 200, 
            "to": 400 
          }, 
          {
    
            "from": 400 
          } 
        ] 
      } 
    } 
  } 
 
#效果 
{
    
  "took" : 131, 
  "timed_out" : false, 
  "_shards" : {
    
    "total" : 5, 
    "successful" : 5, 
    "skipped" : 0, 
    "failed" : 0 
  }, 
  "hits" : {
    
    "total" : {
    
      "value" : 12, 
      "relation" : "eq" 
    }, 
    "max_score" : 1.0, 
    "hits" : [ 
      {
    
        "_index" : "sms-logs-index", 
        "_type" : "_doc", 
        "_id" : "3", 
        "_score" : 1.0, 
        "_source" : {
    
          "createDate" : "2020-09-16", 
          "senDate" : "2020-09-16", 
          "longCode" : "5478434123", 
          "moblie" : 18056587445, 
          "corpName" : "中威集团", 
          "smsContent" : "中威集团，服务于你的身边！", 
          "state" : "0", 
          "opratorId" : "3", 
          "province" : "杭州", 
          "ipAddr" : "10.248.19.45", 
          "replyTotal" : "4", 
          "fee" : "20" 
        } 
      }, 
      {
    
        "_index" : "sms-logs-index", 
        "_type" : "_doc", 
        "_id" : "5", 
        "_score" : 1.0, 
        "_source" : {
    
          "createDate" : "2020-09-16", 
          "senDate" : "2020-09-16", 
          "longCode" : "24514635", 
          "moblie" : 18545427895, 
          "corpName" : "东东集团", 
          "smsContent" : "数据驱动，AI推动，新零售模型让你的购买更心怡！", 
          "state" : "1", 
          "opratorId" : "1", 
          "province" : "北京", 
          "ipAddr" : "10.254.19.45", 
          "replyTotal" : "1", 
          "fee" : "6000" 
        } 
      }, 
      {
    
        "_index" : "sms-logs-index", 
        "_type" : "_doc", 
        "_id" : "11", 
        "_score" : 1.0, 
        "_source" : {
    
          "createDate" : "2020-09-22", 
          "senDate" : "2020-09-22", 
          "longCode" : "458744536", 
          "moblie" : 134625584654, 
          "corpName" : "星雨文化传媒", 
          "smsContent" : "魅力宣传，星雨传媒！", 
          "state" : "1", 
          "opratorId" : "3", 
          "province" : "杭州", 
          "ipAddr" : "10.289.19.45", 
          "replyTotal" : "6", 
          "fee" : "500" 
        } 
      }, 
      {
    
        "_index" : "sms-logs-index", 
        "_type" : "_doc", 
        "_id" : "9", 
        "_score" : 1.0, 
        "_source" : {
    
          "createDate" : "2020-09-16", 
          "senDate" : "2020-09-16", 
          "longCode" : "5784320", 
          "moblie" : 15236964578, 
          "corpName" : "花花派", 
          "smsContent" : "花开花落，魅力女性，买花选我！", 
          "state" : "1", 
          "opratorId" : "1", 
          "province" : "上海", 
          "ipAddr" : "10.265.19.45", 
          "replyTotal" : "1", 
          "fee" : "0.1" 
        } 
      }, 
      {
    
        "_index" : "sms-logs-index", 
        "_type" : "_doc", 
        "_id" : "4", 
        "_score" : 1.0, 
        "_source" : {
    
          "createDate" : "2020-09-16", 
          "senDate" : "2020-09-16", 
          "longCode" : "87454120", 
          "moblie" : 13625789645, 
          "corpName" : "爱美化妆品有限公司", 
          "smsContent" : "魅力，势不可挡，爱美爱美", 
          "state" : "1", 
          "opratorId" : "1", 
          "province" : "上海", 
          "ipAddr" : "10.258.19.45", 
          "replyTotal" : "1", 
          "fee" : "200" 
        } 
      }, 
      {
    
        "_index" : "sms-logs-index", 
        "_type" : "_doc", 
        "_id" : "7", 
        "_score" : 1.0, 
        "_source" : {
    
          "createDate" : "2020-09-16", 
          "senDate" : "2020-09-16", 
          "longCode" : "33656412674", 
          "moblie" : 18956451203, 
          "corpName" : "华丽网集团", 
          "smsContent" : "网络安全，华丽靠谱！", 
          "state" : "1", 
          "opratorId" : "3", 
          "province" : "上海", 
          "ipAddr" : "10.215.254.45", 
          "replyTotal" : "1", 
          "fee" : "2000" 
        } 
      }, 
      {
    
        "_index" : "sms-logs-index", 
        "_type" : "_doc", 
        "_id" : "8", 
        "_score" : 1.0, 
        "_source" : {
    
          "createDate" : "2020-09-16", 
          "senDate" : "2020-09-16", 
          "longCode" : "56412345", 
          "moblie" : 17055452369, 
          "corpName" : "万事Ok公司", 
          "smsContent" : "万事Ok，找我没错！", 
          "state" : "0", 
          "opratorId" : "2", 
          "province" : "杭州", 
          "ipAddr" : "10.215.19.45", 
          "replyTotal" : "1", 
          "fee" : "200" 
        } 
      }, 
      {
    
        "_index" : "sms-logs-index", 
        "_type" : "_doc", 
        "_id" : "10", 
        "_score" : 1.0, 
        "_source" : {
    
          "createDate" : "2020-09-16", 
          "senDate" : "2020-09-16", 
          "longCode" : "54784641", 
          "moblie" : 15625584654, 
          "corpName" : "勾股科技有限公司", 
          "smsContent" : "智能算法，智慧生活，勾股科技！", 
          "state" : "1", 
          "opratorId" : "2", 
          "province" : "杭州", 
          "ipAddr" : "10.215.19.45", 
          "replyTotal" : "6", 
          "fee" : "4000" 
        } 
      }, 
      {
    
        "_index" : "sms-logs-index", 
        "_type" : "_doc", 
        "_id" : "2", 
        "_score" : 1.0, 
        "_source" : {
    
          "createDate" : "2020-09-16", 
          "senDate" : "2020-09-16", 
          "longCode" : "20165411010", 
          "moblie" : 15248754897, 
          "corpName" : "北京鑫鑫能源有限公司", 
          "smsContent" : "欢迎使用新能源，让世界更环保", 
          "state" : "1", 
          "opratorId" : "2", 
          "province" : "北京", 
          "ipAddr" : "10.245.29.280", 
          "replyTotal" : "0.6", 
          "fee" : "0.5" 
        } 
      }, 
      {
    
        "_index" : "sms-logs-index", 
        "_type" : "_doc", 
        "_id" : "6", 
        "_score" : 1.0, 
        "_source" : {
    
          "createDate" : "2020-09-16", 
          "senDate" : "2020-09-16", 
          "longCode" : "89451254", 
          "moblie" : 13028457893, 
          "corpName" : "大兴建筑有限公司", 
          "smsContent" : "我房建，你放心，大兴建筑！", 
          "state" : "1", 
          "opratorId" : "1", 
          "province" : "杭州", 
          "ipAddr" : "10.215.19.45", 
          "replyTotal" : "1", 
          "fee" : "500" 
        } 
      } 
    ] 
  }, 
  "aggregations" : {
    
    "agg" : {
    
      "buckets" : [ 
        {
    
          "key" : "*-200.0", 
          "to" : 200.0, 
          "doc_count" : 4 
        }, 
        {
    
          "key" : "200.0-400.0", 
          "from" : 200.0, 
          "to" : 400.0, 
          "doc_count" : 2 
        }, 
        {
    
          "key" : "400.0-*", 
          "from" : 400.0, 
          "doc_count" : 6 
        } 
      ] 
    } 
  } 
}

时间range统计：实现要求，依据createDate字段创建日期小于2020-09-17和大于等于2020-09-17到至今（2020-10-12）的文档数据分别有多少公司

RESTFUL代码如下；

#时间范围统计date_range 
POST /sms-logs-index/_search 
{
    
  "aggs": {
    
    "agg": {
    
      "date_range": {
    
        "field": "createDate", 
        "format": "yyyy-MM-dd",  
        "ranges": [ 
          {
    
            "to": "2020-09-17" 
          }, 
          {
    
            "from": "2020-09-17", 
            "to": "now" 
          } 
        ] 
      } 
    } 
  } 
}

ip的range统计：实现要求，依据ipAddr字段ip网段小于10.126.2.8和大于等于10.126.2.8的文档数据分别有多少公司

POST /sms-logs-index/_search 
{
    
  "aggs": {
    
    "agg": {
    
      "ip_range": {
    
        "field": "ipAddr", 
        "ranges": [ 
          {
    
            "to": "10.126.2.8" 
          }, 
          {
    
            "from": "10.126.2.8" 
          } 
        ] 
      } 
    } 
  } 
   
}

Java代码如下，这里偷个懒，就写个数值range的Java实现，其他两个都是一样的，大家自己试试吧；

    static RestHighLevelClient myClient= EsClient.getClient();  //获取操作ES的 
    String index="sms-logs-index"; 
 
    @Test 
    public void range() throws IOException 
    {
    
        //1.SearchRequest 
        SearchRequest request=new SearchRequest(index); 
 
        //2.聚合查询条件 
        SearchSourceBuilder builder=new SearchSourceBuilder(); 
        builder.aggregation(AggregationBuilders.range("agg").field("fee").addUnboundedTo(200).addRange(200,400).addUnboundedFrom(400)); 
        request.source(builder); 
 
        //3.执行查询 
        SearchResponse resp = myClient.search(request, RequestOptions.DEFAULT); 
 
        //4.返回结果，默认返回的是父类Aggregation 
        Range agg = resp.getAggregations().get("agg"); 
        for (Range.Bucket bucket : agg.getBuckets()) {
    
            String key=bucket.getKeyAsString(); 
            Object from = bucket.getFrom(); 
            Object to = bucket.getTo(); 
            long docCount = bucket.getDocCount(); 
 
            System.out.println(String.format("key：%s，from：%s，to，%s，docCount：%s",key,from,to,docCount)); 
        } 
    }

Java实现range的效果；

图2 Java实现range聚合统计的效果

extended_stats(统计聚合)查询

可以帮你指定field的最大值，最小值，平均值，平方和等

实现要求，依据fee字段查询最大值，最小值，平均值，平方和等

RESTFUL代码如下；

POST /sms-logs-index/_search 
{
    
  "aggs": {
    
    "aggE": {
    
      "extended_stats": {
    
        "field": "fee" 
      } 
    } 
  } 
}

RESTFUL代码返回结果如下；

{
    
  "took" : 62, 
  "timed_out" : false, 
  "_shards" : {
    
    "total" : 5, 
    "successful" : 5, 
    "skipped" : 0, 
    "failed" : 0 
  }, 
  "hits" : {
    
    "total" : {
    
      "value" : 12, 
      "relation" : "eq" 
    }, 
    "max_score" : 1.0, 
    "hits" : [ 
      {
    
        "_index" : "sms-logs-index", 
        "_type" : "_doc", 
        "_id" : "3", 
        "_score" : 1.0, 
        "_source" : {
    
          "createDate" : "2020-09-16", 
          "senDate" : "2020-09-16", 
          "longCode" : "5478434123", 
          "moblie" : 18056587445, 
          "corpName" : "中威集团", 
          "smsContent" : "中威集团，服务于你的身边！", 
          "state" : "0", 
          "opratorId" : "3", 
          "province" : "杭州", 
          "ipAddr" : "10.248.19.45", 
          "replyTotal" : "4", 
          "fee" : "20" 
        } 
      }, 
      {
    
        "_index" : "sms-logs-index", 
        "_type" : "_doc", 
        "_id" : "5", 
        "_score" : 1.0, 
        "_source" : {
    
          "createDate" : "2020-09-16", 
          "senDate" : "2020-09-16", 
          "longCode" : "24514635", 
          "moblie" : 18545427895, 
          "corpName" : "东东集团", 
          "smsContent" : "数据驱动，AI推动，新零售模型让你的购买更心怡！", 
          "state" : "1", 
          "opratorId" : "1", 
          "province" : "北京", 
          "ipAddr" : "10.254.19.45", 
          "replyTotal" : "1", 
          "fee" : "6000" 
        } 
      }, 
      {
    
        "_index" : "sms-logs-index", 
        "_type" : "_doc", 
        "_id" : "11", 
        "_score" : 1.0, 
        "_source" : {
    
          "createDate" : "2020-09-22", 
          "senDate" : "2020-09-22", 
          "longCode" : "458744536", 
          "moblie" : 134625584654, 
          "corpName" : "星雨文化传媒", 
          "smsContent" : "魅力宣传，星雨传媒！", 
          "state" : "1", 
          "opratorId" : "3", 
          "province" : "杭州", 
          "ipAddr" : "10.289.19.45", 
          "replyTotal" : "6", 
          "fee" : "500" 
        } 
      }, 
      {
    
        "_index" : "sms-logs-index", 
        "_type" : "_doc", 
        "_id" : "9", 
        "_score" : 1.0, 
        "_source" : {
    
          "createDate" : "2020-09-16", 
          "senDate" : "2020-09-16", 
          "longCode" : "5784320", 
          "moblie" : 15236964578, 
          "corpName" : "花花派", 
          "smsContent" : "花开花落，魅力女性，买花选我！", 
          "state" : "1", 
          "opratorId" : "1", 
          "province" : "上海", 
          "ipAddr" : "10.265.19.45", 
          "replyTotal" : "1", 
          "fee" : "0.1" 
        } 
      }, 
      {
    
        "_index" : "sms-logs-index", 
        "_type" : "_doc", 
        "_id" : "4", 
        "_score" : 1.0, 
        "_source" : {
    
          "createDate" : "2020-09-16", 
          "senDate" : "2020-09-16", 
          "longCode" : "87454120", 
          "moblie" : 13625789645, 
          "corpName" : "爱美化妆品有限公司", 
          "smsContent" : "魅力，势不可挡，爱美爱美", 
          "state" : "1", 
          "opratorId" : "1", 
          "province" : "上海", 
          "ipAddr" : "10.258.19.45", 
          "replyTotal" : "1", 
          "fee" : "200" 
        } 
      }, 
      {
    
        "_index" : "sms-logs-index", 
        "_type" : "_doc", 
        "_id" : "7", 
        "_score" : 1.0, 
        "_source" : {
    
          "createDate" : "2020-09-16", 
          "senDate" : "2020-09-16", 
          "longCode" : "33656412674", 
          "moblie" : 18956451203, 
          "corpName" : "华丽网集团", 
          "smsContent" : "网络安全，华丽靠谱！", 
          "state" : "1", 
          "opratorId" : "3", 
          "province" : "上海", 
          "ipAddr" : "10.215.254.45", 
          "replyTotal" : "1", 
          "fee" : "2000" 
        } 
      }, 
      {
    
        "_index" : "sms-logs-index", 
        "_type" : "_doc", 
        "_id" : "8", 
        "_score" : 1.0, 
        "_source" : {
    
          "createDate" : "2020-09-16", 
          "senDate" : "2020-09-16", 
          "longCode" : "56412345", 
          "moblie" : 17055452369, 
          "corpName" : "万事Ok公司", 
          "smsContent" : "万事Ok，找我没错！", 
          "state" : "0", 
          "opratorId" : "2", 
          "province" : "杭州", 
          "ipAddr" : "10.215.19.45", 
          "replyTotal" : "1", 
          "fee" : "200" 
        } 
      }, 
      {
    
        "_index" : "sms-logs-index", 
        "_type" : "_doc", 
        "_id" : "10", 
        "_score" : 1.0, 
        "_source" : {
    
          "createDate" : "2020-09-16", 
          "senDate" : "2020-09-16", 
          "longCode" : "54784641", 
          "moblie" : 15625584654, 
          "corpName" : "勾股科技有限公司", 
          "smsContent" : "智能算法，智慧生活，勾股科技！", 
          "state" : "1", 
          "opratorId" : "2", 
          "province" : "杭州", 
          "ipAddr" : "10.215.19.45", 
          "replyTotal" : "6", 
          "fee" : "4000" 
        } 
      }, 
      {
    
        "_index" : "sms-logs-index", 
        "_type" : "_doc", 
        "_id" : "2", 
        "_score" : 1.0, 
        "_source" : {
    
          "createDate" : "2020-09-16", 
          "senDate" : "2020-09-16", 
          "longCode" : "20165411010", 
          "moblie" : 15248754897, 
          "corpName" : "北京鑫鑫能源有限公司", 
          "smsContent" : "欢迎使用新能源，让世界更环保", 
          "state" : "1", 
          "opratorId" : "2", 
          "province" : "北京", 
          "ipAddr" : "10.245.29.280", 
          "replyTotal" : "0.6", 
          "fee" : "0.5" 
        } 
      }, 
      {
    
        "_index" : "sms-logs-index", 
        "_type" : "_doc", 
        "_id" : "6", 
        "_score" : 1.0, 
        "_source" : {
    
          "createDate" : "2020-09-16", 
          "senDate" : "2020-09-16", 
          "longCode" : "89451254", 
          "moblie" : 13028457893, 
          "corpName" : "大兴建筑有限公司", 
          "smsContent" : "我房建，你放心，大兴建筑！", 
          "state" : "1", 
          "opratorId" : "1", 
          "province" : "杭州", 
          "ipAddr" : "10.215.19.45", 
          "replyTotal" : "1", 
          "fee" : "500" 
        } 
      } 
    ] 
  }, 
  "aggregations" : {
    
    "aggE" : {
    
      "count" : 12, 
      "min" : 0.1, 
      "max" : 6000.0, 
      "avg" : 1160.0583333333334, 
      "sum" : 13920.7, 
      "sum_of_squares" : 5.6830400269999996E7, 
      "variance" : 3390131.3524305555, 
      "std_deviation" : 1841.2309340304262, 
      "std_deviation_bounds" : {
    
        "upper" : 4842.520201394185, 
        "lower" : -2522.403534727519 
      } 
    } 
  } 
}

Java代码如下；

    static RestHighLevelClient myClient= EsClient.getClient();  //获取操作ES的 
    String index="sms-logs-index"; 
 
    @Test 
    public void extended_stats() throws IOException 
    {
    
        //1.SearchRequest 
        SearchRequest request=new SearchRequest(index); 
 
        //2.聚合查询条件 
        SearchSourceBuilder builder=new SearchSourceBuilder(); 
        builder.aggregation(AggregationBuilders.extendedStats("agg").field("fee")); 
        request.source(builder); 
 
        //3.执行查询 
        SearchResponse resp = myClient.search(request, RequestOptions.DEFAULT); 
 
        //4.返回结果，默认返回的是父类Aggregation 
        ExtendedStats agg=resp.getAggregations().get("agg"); 
        double max = agg.getMax(); 
        double min = agg.getMin(); 
 
        //其他的属性就不一一例举了 
 
        System.out.println("max fee :  "+max+"  min fee :  "+min); 
 
    }

图3 Java实现extended_stats聚合统计的效果

聚合统计的函数在ES太丰富了，此处无法一一例举，剩下的留给大家举一反三更多的内容可以参考官网Aggregations;

嵌套字段的聚合统计aggregations查询

有的时候会用到嵌套结构，而且统计数量的时候要用到嵌套结构里面的字段作为判别，如下嵌套结构；

    {
    
        "name": "内容1", 
        "desc": "描述1", 
        "channel": [ 
            {
    
                "name": "one", 
                "num": 28 
            }, 
            {
    
                "name": "two", 
                "num": 32 
            } 
        ] 
    }

为了方便我们对字段中的嵌套类型进行操作Elasticsearch提供了nested方法让我们对嵌套字段实现操作。使用nested需要指定嵌套字段的路径path,假如希望对嵌套字段内的内容进行聚合等操作的时候需要指明字段的全部路径。channel.num;

{
    
    "query": {
    
        "match_all": {
    
            "boost": 1.0 
        } 
    }, 
    "aggregations": {
    
        "test": {
    
            "nested": {
    
                "path": "channel" 
            }, 
            "aggregations": {
    
                "sumChannel": {
    
                    "sum": {
    
                        "field": "channel.num" 
                    } 
                } 
            } 
        } 
    } 
}

结果如下；此时我们对嵌套内数据的求和操作，并不是仅仅将一条数据内此字段的数据进行聚合，而是将所有数据中嵌套内的值求和。下面的返回内容中test桶中的sumChannel的结果就是将两条数据的四个num值进行了相加操作。

{
    
    "took": 133, 
    "timed_out": false, 
    "_shards": {
    
        "total": 1, 
        "successful": 1, 
        "skipped": 0, 
        "failed": 0 
    }, 
    "hits": {
    
        "total": {
    
            "value": 2, 
            "relation": "eq" 
        }, 
        "max_score": 1, 
        "hits": [ 
            {
    
                "_index": "test_field2", 
                "_type": "_doc", 
                "_id": "1", 
                "_score": 1, 
                "_source": {
    
                    "name": "内容1", 
                    "desc": "描述1", 
                    "channel": [ 
                        {
    
                            "name": "one", 
                            "num": 28 
                        }, 
                        {
    
                            "name": "two", 
                            "num": 32 
                        } 
                    ] 
                } 
            }, 
            {
    
                "_index": "test_field2", 
                "_type": "_doc", 
                "_id": "2", 
                "_score": 1, 
                "_source": {
    
                    "name": "内容1", 
                    "desc": "描述1", 
                    "channel": [ 
                        {
    
                            "name": "one", 
                            "num": 33 
                        }, 
                        {
    
                            "name": "two", 
                            "num": 44 
                        } 
                    ] 
                } 
            } 
        ] 
    }, 
    "aggregations": {
    
        "test": {
    
            "doc_count": 4, 
            "sumChannel": {
    
                "value": 137 
            } 
        } 
    } 
}

Java代码如下；

static RestHighLevelClient myClient= EsClient.getClient();  //获取操作ES的 
String index="sms-logs-index"; 
 
public static void queryIndex() throws IOException {
    
        NestedAggregationBuilder nested = AggregationBuilders.nested("test", "channel"); 
        nested.subAggregation(AggregationBuilders.sum("sumChannel").field("channel.num")); 
 
        // match all query 查询所有数据 
        SearchRequest searchRequest = new SearchRequest(); 
        SearchSourceBuilder searchSourceBuilder = new SearchSourceBuilder(); 
        searchSourceBuilder.query(QueryBuilders.matchAllQuery()); 
        searchSourceBuilder.aggregation(nested); 
        searchRequest.source(searchSourceBuilder); 
        // 同步执行 RestClientUtils.client 获得ES连接的方法 
        SearchResponse searchResponse = myClient.search(searchRequest, RequestOptions.DEFAULT); 
        Aggregations aggregations = searchResponse.getAggregations(); 
    }

本文参考链接：https://blog.csdn.net/LXWalaz1s1s/article/details/109002005

IT虾米网搜索引擎

微信公众号号：IT虾米（左侧二维码扫一扫）欢迎添加！

Q : 823053937

热贴推荐

ElasticSearch之cardinality，range，extended_stats聚合统计aggregations查询

ElasticSearch查询笔记目录

整体Java代码的测试用例项目

前言

cardinality(去重计数)查询

range(范围统计)查询

extended_stats(统计聚合)查询

嵌套字段的聚合统计aggregations查询

Q : 823053937

热贴推荐

ElasticSearch之cardinality，range，extended_stats聚合统计aggregations查询

ElasticSearch查询笔记目录

整体Java代码的测试用例项目

前 言

cardinality(去重计数)查询

range(范围统计)查询

extended_stats(统计聚合)查询

嵌套字段的聚合统计aggregations查询

ElasticSearch中嵌套结构使用

Kibana 7.7.1安装

ElasticSearch7.7.1安装分词器——ik分词器和hanlp分词器

ElasticSearch处理数据库里面的null值

ElasticSearch实现字符串既要分词查询又要字段整体排序(一个字段既要text也要keyword)

报警校验ElasticSearch和Hive数据仓库内的数据质量（Java版本）

ElasticSearch之geo_distance，geo_bounding_box，geo_polygon地图检索geo查询

ElasticSearch之scroll，delete-by-query，bool，boosting，filter，highlight查询

ElasticSearch之prefix，fuzzy，wildcard，range，regexp查询

ElasticSearch之term，terms，match，id查询

ElasticSearch7.9.2安装指南之二

ElasticSearch数据结构和基本操作

前言