Quantcast
Channel: Question and Answer » postgresql
Viewing all articles
Browse latest Browse all 1138

To manage pointers to big binary files in Partial index of PostgreSQL?

$
0
0

Assume you have binary data files of size > 1GB.
These files contains events each marked by 585 in hex.
I convert the binary to hex by xxd -ps data.raw | sed ':a;N;$!ba;s/n//g'.
I can get wanted hex position to binary back by |xxd -r.
I characterize characteristics of data in hex about which I propose to create a table called hex but I should have pointers into the binary data directly, since it is too expensive to convert everything to hex – back and forth.
I skip the table hex here.

I think you can get a list of pointers to the start position of each event.
I think the pointer could be stored as VARCHAR but I am not sure if it is the optimal one.

Table

CREATE TABLE events
    (
        event_id SERIAL PRIMARY KEY NOT NULL, 
        measurement_id INTEGER NOT NULL, 
        eventPointer VARCHAR NOT NULL,
        is_active BOOLEAN DEFAULT FALSE
    );

CREATE UNIQUE INDEX dir_events 
    ON events (measurement_id)
    USING btree
    (eventBody)
    WHERE is_active;

CREATE OR REPLACE VIEW events_current AS
    SELECT event_id, eventPointer,
        FROM events
        WHERE is_active 

where the second entry creates the partial index, while the last one is a view to access the index always through the same function.
I think I need event_ID also but I am not sure if you can use pointer as an ID too. I think not.

How can you manage pointers well in partial index?


Viewing all articles
Browse latest Browse all 1138

Latest Images

Trending Articles



Latest Images