Big Database

Name: Big Database
ISBN: 9787121316197

Liu PengPublishing House of Electronics Industry · ChinaJan 2021

3.1

Word count: 140K words
Chapters: 188
ISBN: 9787121316197

Report an error Find this book

Summary

This book provides a detailed introduction to commonly used databases and related tools in big data environments, including HBase, Hive, Impala, Spark, Spark SQL, key-value databases, and streaming databases. It presents detailed working mechanisms and application methods, along with complete practical cases and code.

Editor's review

As a professional work focused on the core technologies of big data storage and processing, Big Database stands out in today's data-driven era. Rather than staying at the conceptual level, it deeply analyzes key technical systems such as distributed file systems, NoSQL databases, and NewSQL databases, with systematic exposition of the architecture, principles, and practical applications of mainstream systems including Hadoop, HBase, MongoDB, and Redis. As a well-known scholar and pioneer in China's cloud computing and big data fields, author Liu Peng combines academic rigor with an engineering practice perspective, ensuring depth and currency.

From an editor's professional judgment, the book's standout strength is its clear hierarchical structure and its tracing of technological evolution. It organizes the vast field of big data storage technologies along a main line that runs from the limitations of traditional relational databases to the rise of distributed file systems, then to the differentiation of various NoSQL databases and the integrative innovation of NewSQL, enabling readers to build a systematic knowledge map. The explanation of core theories such as the CAP theorem, consistency models, data sharding, and replication strategies provides a solid theoretical foundation for understanding the trade-offs behind various technology choices.

In terms of reading value, the book is well suited as a textbook for upper-level undergraduates and graduate students in computer-related majors, or as a reference for engineers and architects in the big data field seeking a deeper understanding of the underlying layers. It helps readers see beyond the surface of popular technology frameworks to understand their design philosophy and applicable scenarios, enabling more reasonable technical decisions in practical work. Although specific technology versions may iterate over time, the core principles and design ideas expounded in the book retain lasting reference value.

Who it's for

A good fit for

Upper-level undergraduates and graduate students in computer science, software engineering, and related majors who want to systematically learn big data storage technology systems
Mid- to senior-level engineers engaged in big data platform development, operations, or architecture design who need to deepen their understanding of underlying principles
Technical team leads or architects seeking theoretical grounding for selecting appropriate data storage solutions for projects
Enthusiasts with a strong interest in distributed systems and database principles who want to go beyond a single technology framework for systematic learning

Not a good fit for

Complete beginners with no programming or database background; the book requires some basic computer knowledge
Business or management personnel who only want a quick overview of big data concepts or application scenarios without delving into technical details

Reading tips

Reading in chapter order is recommended; the first three chapters on big data challenges and distributed fundamentals are especially important as the foundation for later content
While reading, verify your understanding by operating the open-source systems mentioned in the book (such as HBase and MongoDB) using their official documentation or a practice environment
The book can be read alongside Professor Liu Peng's other work, Cloud Computing, to build a complete big data knowledge system from the two dimensions of storage and computation

About the publisher

Publishing House of Electronics Industry China · Founded in 1982

The Publishing House of Electronics Industry is a comprehensive publisher distinguished by its focus on electronic information and related fields. Judging from the more than 3,000 titles in its catalogue, its publishing covers programming and design, artificial intelligence, computer science, finance and economics, photography, and social sciences, attending both to cutting-edge technology and to general readers. Representative titles include Factor Investing: Methods and Practice, x86 Assembly Language: From Real Mode to Protected Mode, The RISC-V Reader, Artificial Intelligence: Speech Recognition, Understanding and Practice, Phased Array Radar Technology, Radar Target Characteristics, and 100 Go Mistakes and How to Avoid Them, most of which centre on technical methods and engineering practice. Its authors include Shi Chuan, Li Zhong, Wang Xiaobo, David Patterson, Andrew Waterman, and Yu Zihao, among others from China and abroad. Owing to limited public information, no further publishing features are elaborated here.

版权信息
内容简介
编写组
总序
前言
第1章大数据库概述
1.1 传统关系型数据库面临的问题
1.2 大数据库技术
1.2.1 列式数据库
1.2.2 内存数据库
1.2.3 键值数据库
1.2.4 流式数据库
1.3 大数据SQL
1.4 当前主流大数据SQL简介
1.4.1 Hive
1.4.2 Impala
1.4.3 Shark
1.4.4 Spark SQL
1.5 本章总结
习题
参考文献
第2章分布式数据库HBase
2.1 HBase基础
2.1.1 体系架构
2.1.2 数据模型
2.2 HBase操作简介
2.2.1 HBase接口简介
2.2.2 HBase Shell实战
2.2.3 HBase API
2.3 HBase实战
2.3.1 实战HBase之综例
2.3.2 实战HBase之使用MapReduce构建索引
习题
参考文献
第3章数据仓库工具Hive
3.1 Hive简介
3.1.1 工作原理
3.1.2 体系架构
3.1.3 计算模型
3.1.4 Hive部署模式
3.2 Hive的使用
3.2.1 Hive的数据类型
3.2.2 Hive接口汇总
3.3 实战Hive Shell
3.3.1 DDL操作
3.3.2 DML操作
3.3.3 SQL操作
3.4 实战Hive之复杂语句
3.5 实战Hive之综合示例
3.6 实战Hive API接口
3.6.1 UDF编程示例
3.6.2 UDAF编程示例
习题
参考文献
第4章大数据查询系统Impala
4.1 Impala简介
4.1.1 Impala的起源
4.1.2 Impala的特点
4.1.3 Impala 前辈及竞争对手
4.2 Impala工作原理
4.2.1 Impala 设计目标
4.2.2 Impala 服务器组件
4.2.3 Impala 编程特点
4.2.4 Impala在Hadoop生态圈中的生存之道
4.3 Impala环境搭建
4.3.1 Impala 安装前的考虑
4.3.2 Impala 安装途径与安装示范
4.4 Impala操作实例
4.4.1 Impala 基本操作
4.4.2 Impala 数据库操作
习题
参考文献
第5章内存数据库Spark
5.1 Spark简介
5.1.1 Spark的引入
5.1.2 Spark生态系统BDAS
5.1.3 Spark系统架构
5.1.4 Spark工作流程
5.1.5 Spark应用案例
5.2 Spark计算模型
5.2.1 Spark程序模型
5.2.2 弹性分布式数据集（RDD）
5.2.3 Spark算子
5.3 Spark工作机制
5.3.1 Spark运行机制
5.3.2 Spark调度机制
5.3.3 Spark I/O机制
5.3.4 Spark通信机制
5.3.5 Spark容错机制
5.3.6 Spark Shuffle机制
5.4 Scala快速入门
5.4.1 Scala解释器
5.4.2 变量
5.4.3 函数
5.4.4 编写Scala脚本
5.4.5 while 配合if实现循环
5.4.6 foreach和 for 来实现迭代
5.4.7 类型参数化数组
5.4.8 Lists
5.4.9 使用元组（Tuples）
5.4.10 Sets和Maps
5.4.11 函数编程风格
5.4.12 读取文件
5.5 Spark环境部署
5.5.1 安装与配置Spark
5.5.2 Intellij IDEA构建Spark开发环境
5.5.3 SBT构建Spark程序
5.5.4 编译Spark程序
5.5.5 远程调试Spark程序
5.5.6 生成Spark部署包
5.6 Spark 编程案例
5.6.1 WordCount
5.6.2 Top K
5.6.3 倒排索引
习题
参考文献
第6章 Spark SQL
6.1 Spark SQL简介
6.1.1 Spark SQL发展历程
6.1.2 Spark SQL 架构
6.2 Spark SQL编程基础
6.2.1 数据类型及表达式
6.2.2 Spark SQL查询引擎Catalyst
6.2.3 SQL DSL API
6.2.4 Spark SQL ThriftServer和CLI
6.2.5 Spark SQL常用操作
6.3 Spark SQL实战
6.3.1 Spark SQL开发环境搭建
6.3.2 Spark SQL使用入门
习题
参考文献
第7章键值数据库
7.1 概述
7.1.1 键值存储
7.1.2 键值数据库
7.2 Redis
7.2.1 简介
7.2.2 Redis数据服务及集群技术
7.2.3 Redis安装
7.2.4 Redis数据操作
7.2.5 案例：网站访问历史记录查询
7.3 Memcached
7.3.1 简介
7.3.2 Memcached缓存技术
7.3.3 Memcached安装
7.3.4 Memcached数据操作
7.3.5 Memcached分布式技术
7.3.6 案例：论坛帖子信息缓存
7.4 典型应用及局限
7.4.1 典型应用
7.4.2 键值数据库局限
习题
参考文献
第8章流式数据库
8.1 流式计算模型
8.1.1 流式计算概念
8.1.2 流式计算数据特点
8.1.3 流式计算典型应用
8.1.4 典型流式计算平台
8.2 流式计算关键技术
8.2.1 计算拓扑
8.2.2 消息传递
8.2.3 高可用性
8.2.4 语义保障
8.2.5 其他关键技术
8.3 Storm平台
8.3.1 Storm简介
8.3.2 Storm原理
8.3.3 Storm部署
8.3.4 案例：Maven环境下的Storm编程
8.4 Spark Streaming平台
8.4.1 Spark Streaming简介
8.4.2 Spark Streaming原理
8.4.3 案例：集群环境下的Spark Streaming编程
习题
参考文献
第9章大数据应用托管平台Docker
9.1 Docker技术简介
9.1.1 Docker是什么
9.1.2 Docker的架构和流程
9.2 Docker的优势和局限
9.2.1 Docker的优势
9.2.2 Docker的局限性
9.3 基于Docker的大数据系统设计
9.3.1 分布式Docker网络环境的搭建
9.3.2 Docker集群管理系统：Kubernetes
习题
参考文献

FAQ

How is this book different from a plain Hadoop or Spark tutorial?

This book focuses on the "storage" layer of the big data ecosystem, systematically explaining the principles of various databases and file systems, which is a deeper foundation. Many tutorials focus on the "computation" and application development of a specific framework (such as Spark). This book aims to help you understand how data is stored and organized, which is a prerequisite for efficiently using computation frameworks.

Are the technologies in the book outdated now?

Specific product versions mentioned in the book will update, but the core principles it explains (such as the CAP theorem, consistency models, distributed transactions, LSM trees, and the Raft protocol) remain unchanged. Understanding these principles enables you to quickly grasp any new technology variant. The book's value lies precisely in conveying these underlying logics.

Is there a lot of math or theoretical derivation? Is it hard to read?

The book focuses on engineering architecture and system design principles rather than pure theoretical derivation. It includes necessary formulas and model explanations, but the overall style leans toward technical exposition rather than mathematical proof. As long as you have basic knowledge of computer systems and data structures, reading is not difficult. The key is some receptiveness to distributed system concepts.

Comments

Find this book

This site is an index: you start “Search all sources” in your browser, and the entries it lists all point to third-party sites; please judge their availability yourself.

About the author

Liu Peng Author · China · 1970

Liu Peng (born 1970) is a Chinese computer scientist. He holds a doctorate in computer architecture from Tsinghua University, has served as a professor and doctoral supervisor at the PLA University of Science and Technology, and is currently chairman of Nanjing Yunchuang Big Data Technology Co., Ltd. He has long been engaged in research and teaching in cloud computing, big data, and artificial intelligence. Fourteen of his works are held in the collection, spanning popular science, artificial intelligence, general computing, medicine, literary appreciation, and agriculture and forestry. His writing ranges from professional textbooks to general introductions, including works on cloud computing, machine learning and deep learning, pattern recognition, and the mathematical foundations of artificial intelligence, as well as a general reader on the Huangdi Neijing and a book on prefabricated structures in waterway engineering. Public information is limited, and details of his life should be based on the records available here.