HyperDex代码走读
代码仓库 | 论文地址 |我就找到这么一份中文资料
[toc]
主要关注点
-
内存表hashtable?如何gc
-
整体交互流程?怎么对标redis的?别是个全内存吧我看到用leveldb了
编译半天编译不过,放弃了,心里路程贴到下面
直接看代码吧
主要流程
组成
Coordinator
Daemon
e::garbage_collector m_gc;
e::garbage_collector::thread_state m_gc_ts;
std::auto_ptr<coordinator_link> m_coord;
std::string m_data_dir;
datalayer m_data;
communication m_comm;
replication_manager m_repl;
state_transfer_manager m_stm;
search_manager m_sm;
(make_obj_func(&daemon::collect_stats, this))
m_gc.register_thread(&m_gc_ts);
m_coord.reset(new coordinator_link(this, coordinator.address.c_str(), coordinator.port));
generate_token(&sid)
m_coord->register_server(server_id(sid), bind_to)
具体的get set在datalayer
datalayer
构成
daemon* m_daemon;
leveldb_db_ptr m_db;
std::vector<index_state> m_indices;
e::ao_hash_map<region_id, uint64_t, id, defaultri> m_versions;
const std::auto_ptr<checkpointer_thread> m_checkpointer;
const std::auto_ptr<wiper_indexer_mediator> m_mediator;
const std::auto_ptr<indexer_thread> m_indexer;
const std::auto_ptr<wiper_thread> m_wiper;
内存version表主要是ao_hash_map,这个设计的非常轻量,因为regionid之类的数据比较少,比较难冲突,问题:有必要单独搞一个hashmap吗?
存储用的hyperleveldb,本来还想看一下hyperleveldb做了啥优化,这里有个测试数据heperleveldb基本被吊锤,所以不看了,就当leveldb
主要设计
Hyperspace Hashing
编译不成功
依赖libpo6和llibe和busybee都是自己写的库,编译都差不多
busybee依赖sparsehash 安装类似
HyperLevelDB是用leveldb改的,得用release/1.2分支
Replicant依赖glog
用的glog版本还有点问题,编译不过
CXXLD replicant-daemon
ld: error: undefined symbol: google::ErrnoLogMessage::ErrnoLogMessage(char const*, int, int, unsigned long, void (google::LogMessage::*)())
>>> referenced by acceptor.cc:288 (daemon/acceptor.cc:288)
>>> daemon/acceptor.o:(replicant::acceptor::garbage_collector::run())
>>> referenced by acceptor.cc:281 (daemon/acceptor.cc:281)
>>> daemon/acceptor.o:(replicant::acceptor::garbage_collector::run())
>>> referenced by daemon.cc:2020 (daemon/daemon.cc:2020)
>>> daemon/daemon.o:(replicant::daemon::rebootstrap(replicant::bootstrap))
>>> referenced 18 more times
>>> did you mean: google::ErrnoLogMessage::ErrnoLogMessage(char const*, int, int, int, void (google::LogMessage::*)())
>>> defined in: /usr/lib/gcc/x86_64-redhat-linux/7/../../../../lib64/libglog.so
collect2: error: ld returned 1 exit status
我把所有代码里的PLOG都删掉就能编译了
依赖libmacaroons-devel可以用安装包工具来安装
libtreadstone 安装方法也是类似的(为什么不放在一起?东一个西一个不难受吗)
还依赖gperf,不是gperftools,gperf Perfect hash function generator
PKG_CONFIG_PATH=$PKG_CONFIG_PATH:/usr/local/lib/pkgconfig
export PKG_CONFIG_PATH
autoreconf -i
automake --add-missing
./configure
编译完libe要拷贝pc文件到pkgconfig目录下
cp libe.pc /usr/local/lib/pkgconfig
cp libpo6.pc /usr/local/lib/pkgconfig
cp libhyperleveldb.pc /usr/local/lib/pkgconfig
cp busybee.pc /usr/local/lib/pkgconfig
然后就是hyperdex各种编译不过提示库没有接口。放弃挣扎了
其他发现
- https://bit-calculator.com/bit-shift-calculator啥稀奇古怪的东西都有啊,1«16 == 65536