IT虾米网

MapReduce之WritableComparable类介绍

flyfish 2022年04月29日 大数据 198 0

WritableComparable类介绍

1.源码

package org.apache.hadoop.io; 
 
import org.apache.hadoop.classification.InterfaceAudience; 
import org.apache.hadoop.classification.InterfaceStability; 
 
@InterfaceAudience.Public 
@InterfaceStability.Stable 
public interface WritableComparable<T> extends Writable, Comparable<T> {
   
     
}

2.简介

A Writable which is also Comparable.  
#译:一个既可比较又可序列化的【类】 
 
WritableComparables can be compared to each other, typically via Comparators.  
Any type which is to be used as a key in the Hadoop Map-Reduce framework  
should implement this interface. 
#译:WritableComparables能够借助比较器实现比较,在Hadoop MapReduce 框架下的被作为key的任何类型都必须 
实现这个接口。 
 
Note that hashCode() is frequently used in Hadoop to partition keys. 
It's important that your implementation of hashCode() returns the same  
result across different instances of the JVM. Note also that the default  
hashCode() implementation in Object does not satisfy this property. 
#译:注意hashCode()经常被用于Hadoop的分区键中。重要的是在实现hashcode()时,在不同的JVM实例中仍需要返回相同的结果【这句话什么意思??】。同时需要注意Object类中默认的hashCode()并不满足这个特性。

虽然Hadoop MapReduce框架为我们提供了很多key类型,但是有些时候,MapReduce过程中的Key很复杂,需要考虑到多个维度的变化,从而导致key并不完全符合Hadoop提供的原生类型,比如说Text(对应Java中的String,)、LongWritable(对应Java中的long)等。所以我们就需要自定义Hadoop MapReduce过程中的key类型。根据这个基类去比较key,我们所需要定义的类必须实现WritableComparable这个类。

  • 01.key是用来进行相互比较,从而达到排序的目的。
  • 02.WritableComparable类实现了Writable(实现的类可序列化) 和Comparable(实现的类可比较)接口。

3.示例

Example: 
     public class MyWritableComparable implements WritableComparable<MyWritableComparable> { 
      // Some data 
      private int counter; 
      private long timestamp; 
 
      public void write(DataOutput out) throws IOException { 
        out.writeInt(counter); 
        out.writeLong(timestamp); 
      } 
 
      public void readFields(DataInput in) throws IOException { 
        counter = in.readInt(); 
        timestamp = in.readLong(); 
      } 
 
      public int compareTo(MyWritableComparable o) { 
        int thisValue = this.value; 
        int thatValue = o.value; 
        return (thisValue &lt; thatValue ? -1 : (thisValue==thatValue ? 0 : 1)); 
      } 
 
      public int hashCode() { 
        final int prime = 31; 
        int result = 1; 
        result = prime * result + counter; 
        result = prime * result + (int) (timestamp ^ (timestamp &gt;&gt;&gt; 32)); 
        return result 
      } 
    }

评论关闭
IT虾米网

微信公众号号:IT虾米 (左侧二维码扫一扫)欢迎添加!

HBASE中常见问题