Record Details

Large Scale Graph Processing in a Distributed Environment

Electronic Theses of Indian Institute of Science

View Archive Info
 
 
Field Value
 
Title Large Scale Graph Processing in a Distributed Environment
 
Creator Upadhyay, Nitesh
 
Subject Distributed Environment
Multi-core Processors
Artificial Intelligence
Computer Network
Scale Graph
Graph Algorithms
Bulk Synchronous Parallel (BSP) Model
Large Scale Graph
CPU Cluster
Giraph
Computer Science
 
Description Graph algorithms are ubiquitously used across domains. They exhibit parallelism, which can be exploited on parallel architectures, such as multi-core processors and accelerators. However, real world graphs are massive in size and cannot fit into the memory of a single machine. Such large graphs are partitioned and processed in a distributed cluster environment which consists of multiple GPUs and CPUs.
Existing frameworks that facilitate large scale graph processing in the distributed cluster have their own style of programming and require extensive involvement by the user in communication and synchronization aspects. Adaptation of these frameworks appears to be an overhead for a programmer. Furthermore, these frameworks have been developed to target only CPU clusters and lack the ability to harness the GPU architecture.
We provide a back-end framework to the graph Domain Specific Language, Falcon, for large scale graph processing on CPU and GPU clusters. The Motivation behind choosing this DSL as a front-end is its shared-memory based imperative programmability feature. Our framework generates Giraph code for CPU clusters. Giraph code runs on the Hadoop cluster and is known for scalable and fault-tolerant graph processing. For GPU cluster, Our framework applies a set of optimizations to reduce computation and communication latency, and generates efficient CUDA code coupled with MPI.
Experimental evaluations show the scalability and performance of our framework for both CPU and GPU clusters. The performance of the framework generated code is comparable to the manual implementations of various algorithms in distributed environments.
 
Contributor Srikant, Y N
 
Date 2018-05-25T15:23:31Z
2018-05-25T15:23:31Z
2018-05-25
2017
 
Type Thesis
 
Identifier http://etd.iisc.ernet.in/2005/3625
http://etd.iisc.ernet.in/abstracts/4495/G28466-Abs.pdf
 
Language en_US
 
Relation G28466