Research Article 
A Job Scheduling Policy based on the Job-classification and Dynamic Replica Mechanism
Yan Qiang, Yue Li, Wei Wei, Bo Pei, Juanjuan Zhao and Hui Zhang
Information Technology Journal, 2014, 13(3), 501-507.
Abstract
In order to guarantee the stability of the system performance and the high Qos(quality of service) of users, a new method based on the HDFS (Hadoop Distributed File System) was proposed which including a job type classification method and a dynamic replica manage mechanism. The method uses the job type classification method to select the I/O intensive job, in order to achieve more accuracy of the classification taken the heterogeneity of the jobs into consideration. For the classified jobs, a dynamic replica manage mechanism was used to determine whether to increase or decrease the number of copies on the specific data node. For a test of a cluster with 1 namenode and 20 data nodes, the method has a high performance. The theoretical and experimental analyses in this paper prove that the proposed method has the ability to improve the performance of HDFS effectively.
ASCI-ID: 28-2222
Similar Articles
Cloud Simulation Resource Scheduling Algorithm Based on Multi-dimension Quality of Service
Information Technology Journal, 2012, 11(1), 94-101.
A Cloud-Based Collaborative Manufacturing Resource Sharing ServicesInformation Technology Journal, 2012, 11(9), 1258-1264.
Integration of SME, Industry and Government Through Public Infrastructure of SOA and Cloud ComputingInformation Technology Journal, 2013, 12(4), 594-603.
Optimal Scheduling for Simulation Resource of Tactical Communication Network Based on Cloud ComputingInformation Technology Journal, 2013, 12(4), 735-741.
A New Massive Data Processing Framework under Cloud Environment for Digital CommunityInformation Technology Journal, 2013, 12(6), 1079-1088.
A User-specific Trusted Virtual Environment for Cloud ComputingInformation Technology Journal, 2013, 12(10), 1905-1913.
Knowledge Management Based on Big Data ProcessingInformation Technology Journal, 2014, 13(7), 1415-1418.
Research on Resource Scheduling Algorithm Based on ForCES NetworkInformation Technology Journal, 2013, 12(12), 2419-2425.
Mobile Social Network Based on SOA and Cloud ComputingInformation Technology Journal, 2014, 13(7), 1439-1442.
A New Type of Center Data Structure in Cloud ComputingInformation Technology Journal, 2014, 13(3), 461-468.
Hybrid Encryption Algorithms in Cloud ComputingInformation Technology Journal, 2013, 12(14), 3015-3019.
An Elastic Meteorological Cloud Platform Based on XenInformation Technology Journal, 2013, 12(15), 3174-3178.
Proposal of Cloud Computing Platform for Enterprise Comprehensive Risk ManagementInformation Technology Journal, 2013, 12(16), 3843-3848.
A new Strategy of Resource Management for Cloud ComputingInformation Technology Journal, 2013, 12(17), 3964-3969.
Library Construction in Nanyang Normal University based on Cloud ComputingInformation Technology Journal, 2013, 12(17), 4272-4276.
A Dynamic Clustering Algorithm for Cloud ComputingInformation Technology Journal, 2013, 12(18), 4637-4641.
Saving Energy in Data Center Networks with Traffic-Aware Virtual Machine PlacementInformation Technology Journal, 2013, 12(19), 5064-5069.
Efficiently Indexing Sparse Wide Tablesin Cloud ComputingInformation Technology Journal, 2013, 12(20), 5415-5423.
A Credible Usage Access Control Model in Cloud ComputingInformation Technology Journal, 2013, 12(20), 5553-5558.
A Multimedia System based on Cloud ComputingInformation Technology Journal, 2013, 12(20), 5656-5660.
Research on Performance Enhancement of Cloud Based on Scheduling and Management ModeInformation Technology Journal, 2013, 12(20), 5885-5890.
Workload Prediction in Load Balancing and Resource Management SystemInformation Technology Journal, 2013, 12(21), 6086-6089.
Research of Main-functional Zones Management Services Cloud Computing PlatformInformation Technology Journal, 2013, 12(21), 6469-6474.
Practical Visual Efficiency Management System for CNC Machine Tools Based on Embedded Microprocessor and Cloud ComputingInformation Technology Journal, 2013, 12(22), 6577-6582.
Service Components-based Resource Allocation in Services CloudInformation Technology Journal, 2013, 12(22), 6676-6680.
The Security Issue of WSNs Based on Cloud Computing for Smart GridInformation Technology Journal, 2013, 12(22), 6702-6709.
A Novel Improved Data Load Balancing Algorithm for HadoopInformation Technology Journal, 2013, 12(23), 7260-7264.
Research on the Security of Virtual Network with Xen PlatformInformation Technology Journal, 2013, 12(23), 7774-7777.
Scientific Data Processing Using Mapreduce in Cloud EnvironmentsInformation Technology Journal, 2013, 12(23), 7869-7873.
A Distributed Algorithm for SI Transactions Serializability in Cloud ComputingInformation Technology Journal, 2013, 12(24), 8435-8445.
Optimized Task Scheduling and Resource Allocation in Cloud Computing Using PSO based Fitness FunctionInformation Technology Journal, 2013, 12(23), 7090-7095.
Evaluation of OpenStack (Havana Release) and CloudStack (4.3 Release) Open Source Cloud SolutionsInformation Technology Journal, 2014, 13(16), 2508-2513.
Making a Smart Government: Lessons from Government Cloud DevelopmentInformation Technology Journal, 2016, 15(1), 18-25.