A single node server for simple document key value store

Ishaan Aggarwal (ishaan.aggarwal@sjsu.edu)

Purpose:

The aim of this deliverable was to get a hands-on with rust by starting with the implementation of a simple single node server which can act as a document key-value store. This will be utilized in future as ultimately, we intend to create an efficient, robust, and high-performance document store.

Key-Value store implementation:

The code utilizes a rust library(called as rust crate) - unqlite. UnQLite is a software library which implements a self-contained, serverless, zero-configuration, transactional NoSQL database engine. UnQLite is a document store database similar to [MongoDB], [Redis], [CouchDB] etc. as well as a standard Key/Value store similar to [BerkeleyDB], [LevelDB], etc. It is an embedded NoSQL (Key/Value store and Document-store) database engine (Reference provided for more information). Using this, I create a key-value store on disk and try storing some key-value pairs. Then code then iterates over those stored values and based on some comparison, it deletes the entries. At the end, it returns the remaining pairs in the store.
This implementation shows the capability of storing/deleting/modifying the entries in the key-value store which will be handy for the ultimate aim of the project which is storing warc files and playing with them.

Code: Key-Value store

main.rs


extern crate unqlite;

use unqlite::{UnQLite, KV, Cursor};

fn main() {
    let unqlite = UnQLite::create_temp();
    // Any type that can use as `[u8]`
    unqlite.kv_store("key1", "a long length value").unwrap();
    unqlite.kv_store("abc", [1,2,3]).unwrap();
    unqlite.kv_store("key2", "value2").unwrap();
    unqlite.kv_store("key3", "value length greater than 10");

    let mut entry = unqlite.first();
    // Iterate records
    loop {
        if entry.is_none() { break; }

        let record = entry.expect("valid entry");
        let (key, value) = record.key_value();
        println!("* Go through {:?} --> {:?}", key, value);

        if value.len() > 10 {
            entry = record.delete();
            println!("** Delete key {:?} due to  value length", key);
        } else {
            entry = record.next();
        }
    }

    //Iterate final records and print each of them
    entry = unqlite.first();
    loop{
        if entry.is_none() { break; }
        let record = entry.expect("valid entry");
        let (key, value) = record.key_value();
        println!("*** Final key-value pairs in store: {:?} --> {:?}", key, value);
        entry = record.next();
    }
}

Warming up to this deliverable:

Initially, I implemented a simple TCP echo server. This server would listen to a specific port for TCP connection and echo back the same message to the client. The client would then verify whether the message it sent was echoed back by the server or not. If yes, it returns success and closes the connection, else returns an error. The code is given below.

Code: TCP server-client

rust_server.rs


use std::thread;
use std::net::{TcpListener, TcpStream, Shutdown};
use std::io::{Read, Write};

fn handle_client(mut stream: TcpStream) {
    let mut data = [0 as u8; 50]; // using 50 byte buffer
    while match stream.read(&mut data) {
        Ok(size) => {
            // echo everything!
            stream.write(&data[0..size]).unwrap();
            true
        },
        Err(_) => {
            println!("An error occurred, terminating connection with {}", stream.peer_addr().unwrap());
            stream.shutdown(Shutdown::Both).unwrap();
            false }
    } {}
}

pub fn main() {
    let listener = TcpListener::bind("0.0.0.0:3333").unwrap();
    // accept connections and process them, spawning a new thread for each one
    println!("Server listening on port 3333");
    for stream in listener.incoming() {
        match stream {
            Ok(stream) => {
                println!("New incoming connection from: {}", stream.peer_addr().unwrap());
                thread::spawn(move|| {
                    // connection succeeded
                    handle_client(stream)
                });
            }
            Err(e) => {
                println!("Error: {}", e);
                /* connection failed */
            }
        }
    }
    // close the socket server
    drop(listener);
}

rust_client.rs


use std::net::{TcpStream};
use std::io::{Read, Write};
use std::str::from_utf8;

pub fn main() {
    match TcpStream::connect("localhost:3333") {
        Ok(mut stream) => {
            println!("Successfully connected to server in port 3333");

            let msg = b"Hello!";
            stream.write(msg).unwrap();
            println!("Sent Hello, awaiting reply...");

            let mut data = [0 as u8; 6]; // using 6 byte buffer
            match stream.read_exact(&mut data) {
                Ok(_) => {
                    if &data == msg {
                        println!("Reply is ok!");
                    } else {
                        let text = from_utf8(&data).unwrap();
                        println!("Unexpected reply: {}", text);
                    }
                },
                Err(e) => {
                    println!("Failed to receive data: {}", e);
                }
            }
        },
        Err(e) => {
            println!("Failed to connect: {}", e);
        }
    }
    println!("Terminated.");
}

References:

[1] Documentation of rust crate - unqlite: https://crates.io/crates/unqlite