Module roaringbitmap

Package org.roaringbitmap.longlong

Class Roaring64Bitmap

java.lang.Object

org.roaringbitmap.longlong.Roaring64Bitmap

All Implemented Interfaces:

Externalizable, Serializable, ImmutableLongBitmapDataProvider, LongBitmapDataProvider

public class Roaring64Bitmap
extends Object
implements Externalizable, LongBitmapDataProvider

Roaring64Bitmap is a compressed 64 bit bitmap. It can contain all the numbers of long rang[Long.MAX_VALUE, Long.MIN_VALUE]. Since java has no unsigned long,we treat the negative value as a successor of the positive value. So the ascending ordering of all the long value is: 0,1,2...Long.MAX_VALUE,Long.MIN_VALUE,Long.MIN_VALUE+1.......-1. See Long.toUnsignedString()

See Also:

• Serialized Form

Nested Class Summary

Nested classes/interfaces inherited from interface org.roaringbitmap.longlong.ImmutableLongBitmapDataProvider

ImmutableLongBitmapDataProvider.RoaringOfLong

Constructor Summary

Constructors

Constructor	Description
Roaring64Bitmap()

Method Summary

Modifier and Type	Method	Description
void	add(long... dat)	Set all the specified values to true.
void	add(long rangeStart, long rangeEnd)	Deprecated. as this may be confused with adding individual longs
void	addInt(int x)
void	addLong(long x)	Add the value to the container (set the value to "true"), whether it already appears or not.
void	addRange(long rangeStart, long rangeEnd)	Add to the current bitmap all longs in [rangeStart,rangeEnd).
void	and(Roaring64Bitmap x2)	In-place bitwise AND (intersection) operation.
static Roaring64Bitmap	and(Roaring64Bitmap x1, Roaring64Bitmap x2)	Bitwise AND (intersection) operation.
static long	andCardinality(Roaring64Bitmap x1, Roaring64Bitmap x2)	Cardinality of Bitwise AND (intersection) operation.
void	andNot(Roaring64Bitmap x2)	In-place bitwise ANDNOT (difference) operation.
static Roaring64Bitmap	andNot(Roaring64Bitmap x1, Roaring64Bitmap x2)	Bitwise ANDNOT (difference) operation.
static Roaring64Bitmap	bitmapOf(long... dat)	Generate a bitmap with the specified values set to true.
void	clear()	reset to an empty bitmap; result occupies as much space a newly created bitmap.
Roaring64Bitmap	clone()
boolean	contains(long x)	Checks whether the value in included, which is equivalent to checking if the corresponding bit is set (get in BitSet class).
void	deserialize(DataInput in)	Deserialize (retrieve) this bitmap.
void	deserialize(ByteBuffer in)	Deserialize (retrieve) this bitmap.
boolean	equals(Object obj)
long	first()	Get the first (smallest) integer in this RoaringBitmap, that is, returns the minimum of the set.
void	flip(long x)	Add the value if it is not already present, otherwise remove it.
void	flip(long rangeStart, long rangeEnd)	Complements the bits in the given range, from rangeStart (inclusive) rangeEnd (exclusive).
void	forAllInRange(long start, int length, RelativeRangeConsumer rrc)	Consume presence information for all values in the range [start, start + length).
void	forEach(LongConsumer lc)	Visit all values in the bitmap and pass them to the consumer.
void	forEachInRange(long start, int length, LongConsumer lc)	Consume each value present in the range [start, start + length).
int	getIntCardinality()
long	getLongCardinality()	Returns the number of distinct integers added to the bitmap (e.g., number of bits set).
PeekableLongIterator	getLongIterator()	For better performance, consider the Use the forEach method.
PeekableLongIterator	getLongIteratorFrom(long minval)	Produce an iterator over the values in this bitmap starting from `minval`.
long	getLongSizeInBytes()	Estimate of the memory usage of this data structure.
PeekableLongIterator	getReverseLongIterator()
PeekableLongIterator	getReverseLongIteratorFrom(long maxval)	Produce an iterator over the values in this bitmap starting from `maxval`.
int	getSizeInBytes()	Estimate of the memory usage of this data structure.
int	hashCode()
static boolean	intersects(Roaring64Bitmap x1, Roaring64Bitmap x2)	Checks whether the two bitmaps intersect.
boolean	isEmpty()	Checks whether the bitmap is empty.
Iterator<Long>	iterator()	For better performance, consider the Use the forEach method.
long	last()	Get the last (largest) integer in this RoaringBitmap, that is, returns the maximum of the set.
ImmutableLongBitmapDataProvider	limit(long x)	Create a new bitmap of the same class, containing at most maxcardinality integers.
void	or(Roaring64Bitmap x2)	In-place bitwise OR (union) operation.
static Roaring64Bitmap	or(Roaring64Bitmap x1, Roaring64Bitmap x2)	Bitwise OR (union) operation.
long	rankLong(long id)	Rank returns the number of integers that are smaller or equal to x (Rank(infinity) would be GetCardinality()).
void	readExternal(ObjectInput in)
void	remove(long x)	If present remove the specified integer (effectively, sets its bit value to false)
void	removeLong(long x)	If present remove the specified integers (effectively, sets its bit value to false)
boolean	runOptimize()	Use a run-length encoding where it is estimated as more space efficient
long	select(long j)	Return the jth value stored in this bitmap.
void	serialize(DataOutput out)	Serialize this bitmap.
void	serialize(ByteBuffer byteBuffer)	Serialize this bitmap, please make sure the size of the serialized bytes is smaller enough that ByteBuffer can hold it.
long	serializedSizeInBytes()	Report the number of bytes required to serialize this bitmap.
long[]	toArray()	Return the set values as an array, if the cardinality is smaller than 2147483648.
String	toString()	A string describing the bitmap.
void	trim()	remove the allocated unused memory space
void	writeExternal(ObjectOutput out)	Roaring64NavigableMap are serializable.
void	xor(Roaring64Bitmap x2)	In-place bitwise XOR (symmetric difference) operation.
static Roaring64Bitmap	xor(Roaring64Bitmap x1, Roaring64Bitmap x2)	Bitwise XOR (symmetric difference) operation.

Methods inherited from class java.lang.Object

finalize, getClass, notify, notifyAll, wait, wait, wait

Methods inherited from interface org.roaringbitmap.longlong.ImmutableLongBitmapDataProvider

reverseStream, stream

Constructor Details

Roaring64Bitmap

public Roaring64Bitmap()

Method Details

addInt

public void addInt(int x)

addLong

public void addLong(long x)

Add the value to the container (set the value to "true"), whether it already appears or not. Java lacks native unsigned longs but the x argument is considered to be unsigned. Within bitmaps, numbers are ordered according toLong.toUnsignedString(long, int). We order the numbers like 0, 1, ..., 9223372036854775807, -9223372036854775808, -9223372036854775807,..., -1.

Specified by:

addLong in interface LongBitmapDataProvider

Parameters:

x - long value

getLongCardinality

public long getLongCardinality()

Returns the number of distinct integers added to the bitmap (e.g., number of bits set).

Specified by:

getLongCardinality in interface ImmutableLongBitmapDataProvider

Returns:

the cardinality

getIntCardinality

public int getIntCardinality()
                      throws UnsupportedOperationException

Returns:

the cardinality as an int

Throws:

UnsupportedOperationException - if the cardinality does not fit in an int

select

public long select(long j)
            throws IllegalArgumentException

Return the jth value stored in this bitmap.

Specified by:

select in interface ImmutableLongBitmapDataProvider

Parameters:

j - index of the value

Returns:

the value

Throws:

IllegalArgumentException - if j is out of the bounds of the bitmap cardinality

first

public long first()

Get the first (smallest) integer in this RoaringBitmap, that is, returns the minimum of the set.

Specified by:

first in interface ImmutableLongBitmapDataProvider

Returns:

the first (smallest) integer

Throws:

NoSuchElementException - if empty

last

public long last()

Get the last (largest) integer in this RoaringBitmap, that is, returns the maximum of the set.

Specified by:

last in interface ImmutableLongBitmapDataProvider

Returns:

the last (largest) integer

Throws:

NoSuchElementException - if empty

iterator

public Iterator<Long> iterator()

For better performance, consider the Use the forEach method.

Returns:

a custom iterator over set bits, the bits are traversed in ascending sorted order

forEach

public void forEach(LongConsumer lc)

Description copied from interface: ImmutableLongBitmapDataProvider

Visit all values in the bitmap and pass them to the consumer. * Usage:

 
  bitmap.forEach(new LongConsumer() {

    {@literal @}Override
    public void accept(long value) {
      // do something here

    }});
   
 }
 

Specified by:

forEach in interface ImmutableLongBitmapDataProvider

Parameters:

lc - the consumer

forAllInRange

public void forAllInRange(long start,
 int length,
 RelativeRangeConsumer rrc)

Consume presence information for all values in the range [start, start + length).

Parameters:

start - Lower bound of values to consume.

length - Maximum number of values to consume.

rrc - Code to be executed for each present or absent value.

forEachInRange

public void forEachInRange(long start,
 int length,
 LongConsumer lc)

Consume each value present in the range [start, start + length).

Parameters:

start - Lower bound of values to consume.

length - Maximum number of values to consume.

lc - Code to be executed for each present value.

rankLong

public long rankLong(long id)

Description copied from interface: ImmutableLongBitmapDataProvider

Rank returns the number of integers that are smaller or equal to x (Rank(infinity) would be GetCardinality()). The value is a full 64-bit value.

Specified by:

rankLong in interface ImmutableLongBitmapDataProvider

Parameters:

id - upper limit

Returns:

the rank

or

public void or(Roaring64Bitmap x2)

In-place bitwise OR (union) operation. The current bitmap is modified.

Parameters:

x2 - other bitmap

or

public static Roaring64Bitmap or(Roaring64Bitmap x1,
 Roaring64Bitmap x2)

Bitwise OR (union) operation. The provided bitmaps are *not* modified. This operation is thread-safe as long as the provided bitmaps remain unchanged.

Parameters:

x1 - first bitmap

x2 - other bitmap

Returns:

result of the operation

xor

public void xor(Roaring64Bitmap x2)

In-place bitwise XOR (symmetric difference) operation. The current bitmap is modified.

Parameters:

x2 - other bitmap

xor

public static Roaring64Bitmap xor(Roaring64Bitmap x1,
 Roaring64Bitmap x2)

Bitwise XOR (symmetric difference) operation. The provided bitmaps are *not* modified. This operation is thread-safe as long as the provided bitmaps remain unchanged.

Parameters:

x1 - first bitmap

x2 - other bitmap

Returns:

result of the operation

and

public void and(Roaring64Bitmap x2)

In-place bitwise AND (intersection) operation. The current bitmap is modified.

Parameters:

x2 - other bitmap

and

public static Roaring64Bitmap and(Roaring64Bitmap x1,
 Roaring64Bitmap x2)

Bitwise AND (intersection) operation. The provided bitmaps are *not* modified. This operation is thread-safe as long as the provided bitmaps remain unchanged.

Parameters:

x1 - first bitmap

x2 - other bitmap

Returns:

result of the operation

intersects

public static boolean intersects(Roaring64Bitmap x1,
 Roaring64Bitmap x2)

Checks whether the two bitmaps intersect. This can be much faster than calling "and" and checking the cardinality of the result.

Parameters:

x1 - first bitmap

x2 - other bitmap

Returns:

true if they intersect

andCardinality

public static long andCardinality(Roaring64Bitmap x1,
 Roaring64Bitmap x2)

Cardinality of Bitwise AND (intersection) operation. The provided bitmaps are *not* modified. This operation is thread-safe as long as the provided bitmaps remain unchanged.

Parameters:

x1 - first bitmap

x2 - other bitmap

Returns:

as if you did and(x1,x2).getCardinality()

andNot

public void andNot(Roaring64Bitmap x2)

In-place bitwise ANDNOT (difference) operation. The current bitmap is modified.

Parameters:

x2 - other bitmap

andNot

public static Roaring64Bitmap andNot(Roaring64Bitmap x1,
 Roaring64Bitmap x2)

Bitwise ANDNOT (difference) operation. The provided bitmaps are *not* modified. This operation is thread-safe as long as the provided bitmaps remain unchanged.

Parameters:

x1 - first bitmap

x2 - other bitmap

Returns:

result of the operation

flip

public void flip(long rangeStart,
 long rangeEnd)

Complements the bits in the given range, from rangeStart (inclusive) rangeEnd (exclusive). The given bitmap is unchanged.

Parameters:

rangeStart - inclusive beginning of range, in [0, 0xffffffffffffffff]

rangeEnd - exclusive ending of range, in [0, 0xffffffffffffffff + 1]

writeExternal

public void writeExternal(ObjectOutput out)
                   throws IOException

Roaring64NavigableMap are serializable. However, contrary to RoaringBitmap, the serialization format is not well-defined: for now, it is strongly coupled with Java standard serialization. Just like the serialization may be incompatible between various Java versions, Roaring64NavigableMap are subject to incompatibilities. Moreover, even on a given Java versions, the serialization format may change from one RoaringBitmap version to another

Specified by:

writeExternal in interface Externalizable

Throws:

IOException

readExternal

public void readExternal(ObjectInput in)
                  throws IOException

Specified by:

readExternal in interface Externalizable

Throws:

IOException

toString

public String toString()

A string describing the bitmap.

Overrides:

toString in class Object

Returns:

the string

getLongIterator

public PeekableLongIterator getLongIterator()

For better performance, consider the Use the forEach method.

Specified by:

getLongIterator in interface ImmutableLongBitmapDataProvider

Returns:

a custom iterator over set bits, the bits are traversed in ascending sorted order

getLongIteratorFrom

public PeekableLongIterator getLongIteratorFrom(long minval)

Produce an iterator over the values in this bitmap starting from `minval`.

Parameters:

minval - the lower bound of the iterator returned

Returns:

a custom iterator over set bits, the bits are traversed in ascending sorted order

contains

public boolean contains(long x)

Description copied from interface: ImmutableLongBitmapDataProvider

Checks whether the value in included, which is equivalent to checking if the corresponding bit is set (get in BitSet class).

Specified by:

contains in interface ImmutableLongBitmapDataProvider

Parameters:

x - long value

Returns:

whether the long value is included.

getSizeInBytes

public int getSizeInBytes()

Description copied from interface: ImmutableLongBitmapDataProvider

Estimate of the memory usage of this data structure. Internally, this is computed as a 64-bit counter.

Specified by:

getSizeInBytes in interface ImmutableLongBitmapDataProvider

Returns:

estimated memory usage.

getLongSizeInBytes

public long getLongSizeInBytes()

Estimate of the memory usage of this data structure. This can be expected to be within 1% of the true memory usage in common usage scenarios. If exact measures are needed, we recommend using dedicated libraries such as ehcache-sizeofengine. In adversarial cases, this estimate may be 10x the actual memory usage. For example, if you insert a single random value in a bitmap, then over a 100 bytes may be used by the JVM whereas this function may return an estimate of 32 bytes. The same will be true in the "sparse" scenario where you have a small set of random-looking integers spanning a wide range of values. These are considered adversarial cases because, as a general rule, if your data looks like a set of random integers, Roaring bitmaps are probably not the right data structure. Note that you can serialize your Roaring Bitmaps to disk and then construct ImmutableRoaringBitmap instances from a ByteBuffer. In such cases, the Java heap usage will be significantly less than what is reported. If your main goal is to compress arrays of integers, there are other libraries that are maybe more appropriate such as JavaFastPFOR. Note, however, that in general, random integers (as produced by random number generators or hash functions) are not compressible. Trying to compress random data is an adversarial use case.

Specified by:

getLongSizeInBytes in interface ImmutableLongBitmapDataProvider

Returns:

estimated memory usage.

See Also:

• JavaFastPFOR

isEmpty

public boolean isEmpty()

Description copied from interface: ImmutableLongBitmapDataProvider

Checks whether the bitmap is empty.

Specified by:

isEmpty in interface ImmutableLongBitmapDataProvider

Returns:

true if this bitmap contains no set bit

limit

public ImmutableLongBitmapDataProvider limit(long x)

Description copied from interface: ImmutableLongBitmapDataProvider

Create a new bitmap of the same class, containing at most maxcardinality integers.

Specified by:

limit in interface ImmutableLongBitmapDataProvider

Parameters:

x - maximal cardinality

Returns:

a new bitmap with cardinality no more than maxcardinality

runOptimize

public boolean runOptimize()

Use a run-length encoding where it is estimated as more space efficient

Returns:

whether a change was applied

serialize

public void serialize(DataOutput out)
               throws IOException

Serialize this bitmap. Unlike RoaringBitmap, there is no specification for now: it may change from one java version to another, and from one RoaringBitmap version to another. Consider calling runOptimize() before serialization to improve compression. The current bitmap is not modified.

Specified by:

serialize in interface ImmutableLongBitmapDataProvider

Parameters:

out - the DataOutput stream

Throws:

IOException - Signals that an I/O exception has occurred.

serialize

public void serialize(ByteBuffer byteBuffer)
               throws IOException

Serialize this bitmap, please make sure the size of the serialized bytes is smaller enough that ByteBuffer can hold it.

Parameters:

byteBuffer - the ByteBuffer

Throws:

IOException - Signals that an I/O exception has occurred.

deserialize

public void deserialize(DataInput in)
                 throws IOException

Deserialize (retrieve) this bitmap. Unlike RoaringBitmap, there is no specification for now: it may change from one java version to another, and from one RoaringBitmap version to another. The current bitmap is overwritten.

Parameters:

in - the DataInput stream

Throws:

IOException - Signals that an I/O exception has occurred.

deserialize

public void deserialize(ByteBuffer in)
                 throws IOException

Deserialize (retrieve) this bitmap. Unlike RoaringBitmap, there is no specification for now: it may change from one java version to another, and from one RoaringBitmap version to another. The current bitmap is overwritten.

Parameters:

in - the ByteBuffer stream

Throws:

IOException - Signals that an I/O exception has occurred.

serializedSizeInBytes

public long serializedSizeInBytes()

Description copied from interface: ImmutableLongBitmapDataProvider

Report the number of bytes required to serialize this bitmap. This is the number of bytes written out when using the serialize method. When using the writeExternal method, the count will be higher due to the overhead of Java serialization.

Specified by:

serializedSizeInBytes in interface ImmutableLongBitmapDataProvider

Returns:

the size in bytes

clear

public void clear()

reset to an empty bitmap; result occupies as much space a newly created bitmap.

toArray

public long[] toArray()

Return the set values as an array, if the cardinality is smaller than 2147483648. The long values are in sorted order.

Specified by:

toArray in interface ImmutableLongBitmapDataProvider

Returns:

array representing the set values.

bitmapOf

public static Roaring64Bitmap bitmapOf(long... dat)

Generate a bitmap with the specified values set to true. The provided longs values don't have to be in sorted order, but it may be preferable to sort them from a performance point of view.

Parameters:

dat - set values

Returns:

a new bitmap

remove

public void remove(long x)

If present remove the specified integer (effectively, sets its bit value to false)

Parameters:

x - integer value representing the index in a bitmap

add

public void add(long... dat)

Set all the specified values to true. This can be expected to be slightly faster than calling "add" repeatedly. The provided integers values don't have to be in sorted order, but it may be preferable to sort them from a performance point of view.

Parameters:

dat - set values

add

@Deprecated
public void add(long rangeStart,
 long rangeEnd)

Deprecated.

as this may be confused with adding individual longs

Add to the current bitmap all longs in [rangeStart,rangeEnd).

Parameters:

rangeStart - inclusive beginning of range

rangeEnd - exclusive ending of range

addRange

public void addRange(long rangeStart,
 long rangeEnd)

Add to the current bitmap all longs in [rangeStart,rangeEnd).

Parameters:

rangeStart - inclusive beginning of range

rangeEnd - exclusive ending of range

getReverseLongIterator

public PeekableLongIterator getReverseLongIterator()

Specified by:

getReverseLongIterator in interface ImmutableLongBitmapDataProvider

Returns:

a custom iterator over set bits, the bits are traversed in descending sorted order

getReverseLongIteratorFrom

public PeekableLongIterator getReverseLongIteratorFrom(long maxval)

Produce an iterator over the values in this bitmap starting from `maxval`.

Parameters:

maxval - the upper bound of the iterator returned

Returns:

a custom iterator over set bits, the bits are traversed in descending sorted order

removeLong

public void removeLong(long x)

Description copied from interface: LongBitmapDataProvider

If present remove the specified integers (effectively, sets its bit value to false)

Specified by:

removeLong in interface LongBitmapDataProvider

Parameters:

x - long value representing the index in a bitmap

trim

public void trim()

remove the allocated unused memory space

Specified by:

trim in interface LongBitmapDataProvider

hashCode

public int hashCode()

Overrides:

hashCode in class Object

equals

public boolean equals(Object obj)

Overrides:

equals in class Object

flip

public void flip(long x)

Add the value if it is not already present, otherwise remove it.

Parameters:

x - long value

clone

public Roaring64Bitmap clone()

Overrides:

clone in class Object