public class NodeSorter extends Object implements INodeSorter
| Modifier and Type | Class and Description |
|---|---|
static interface |
NodeSorter.ExistingScheduleFunc
interface for calculating the number of existing executors scheduled on a object (rack or node).
|
| Modifier and Type | Field and Description |
|---|---|
protected Cluster |
cluster |
protected List<String> |
favoredNodeIds |
protected List<String> |
greyListedSupervisorIds |
protected BaseResourceAwareStrategy.NodeSortType |
nodeSortType |
protected TopologyDetails |
topologyDetails |
protected List<String> |
unFavoredNodeIds |
| Constructor and Description |
|---|
NodeSorter(Cluster cluster,
TopologyDetails topologyDetails,
BaseResourceAwareStrategy.NodeSortType nodeSortType)
Initialize for the default implementation node sorting.
|
| Modifier and Type | Method and Description |
|---|---|
List<ObjectResourcesItem> |
getSortedRacks()
Racks are sorted by two criteria.
|
List<RasNode> |
hostnameToNodes(String hostname)
hostname to Ids.
|
protected List<String> |
makeHostToNodeIds(List<String> hosts) |
void |
prepare(ExecutorDetails exec)
Prepare for node sorting.
|
Iterable<String> |
sortAllNodes() |
protected List<ObjectResourcesItem> |
sortObjectResources(ObjectResourcesSummary resourcesSummary,
ExecutorDetails exec,
NodeSorter.ExistingScheduleFunc existingScheduleFunc)
Scheduling uses
sortAllNodes() which eventually calls this method whose behavior can be altered by setting nodeSortType. |
protected final BaseResourceAwareStrategy.NodeSortType nodeSortType
protected Cluster cluster
protected TopologyDetails topologyDetails
public NodeSorter(Cluster cluster, TopologyDetails topologyDetails, BaseResourceAwareStrategy.NodeSortType nodeSortType)
Initialize for the default implementation node sorting.
BaseResourceAwareStrategy.NodeSortType#GENERIC_RAS sorting implemented in
sortObjectResourcesGeneric(ObjectResourcesSummary, ExecutorDetails, NodeSorter.ExistingScheduleFunc)BaseResourceAwareStrategy.NodeSortType#DEFAULT_RAS sorting implemented in
sortObjectResourcesDefault(ObjectResourcesSummary, NodeSorter.ExistingScheduleFunc)BaseResourceAwareStrategy.NodeSortType#COMMON sorting implemented in
sortObjectResourcesCommon(ObjectResourcesSummary, ExecutorDetails, NodeSorter.ExistingScheduleFunc)cluster - for which nodes will be sorted.topologyDetails - the topology to sort for.nodeSortType - type of sorting to be applied to object resource collection BaseResourceAwareStrategy.NodeSortType.public void prepare(ExecutorDetails exec)
INodeSorterPrepare for node sorting. This method must be called before INodeSorter.getSortedRacks() and INodeSorter.sortAllNodes().
prepare in interface INodeSorterexec - optional, may be null.protected List<ObjectResourcesItem> sortObjectResources(ObjectResourcesSummary resourcesSummary, ExecutorDetails exec, NodeSorter.ExistingScheduleFunc existingScheduleFunc)
Scheduling uses sortAllNodes() which eventually calls this method whose behavior can be altered by setting nodeSortType.
resourcesSummary - contains all individual ObjectResourcesItem as well as cumulative statsexec - executor for which the sorting is doneexistingScheduleFunc - a function to get existing executors already scheduled on this objectObjectResourcesItempublic Iterable<String> sortAllNodes()
sortAllNodes in interface INodeSorterpublic List<ObjectResourcesItem> getSortedRacks()
Racks are sorted by two criteria.
1) the number executors of the topology that needs to be scheduled is already on the rack in descending order. The reasoning to sort based on criterion 1 is so we schedule the rest of a topology on the same rack as the existing executors of the topology.
2) the subordinate/subservient resource availability percentage of a rack in descending order We calculate the resource availability percentage by dividing the resource availability on the rack by the resource availability of the entire cluster By doing this calculation, racks that have exhausted or little of one of the resources mentioned above will be ranked after racks that have more balanced resource availability. So we will be less likely to pick a rack that have a lot of one resource but a low amount of another.
getSortedRacks in interface INodeSorterCopyright © 2021 The Apache Software Foundation. All rights reserved.