我有一个将异步任务委托给线程池的进程。我需要确保某些任务按顺序执行。 因此,例如使用ExecutorService控制任务执行顺序
任务按顺序到达
任务A1,B1,C1,D1,E1,A2,A3,B2,F1
任务可以以任何顺序,除非有一个被执行自然依赖,所以a1,a2,a3必须按照这个顺序处理,或者分配给同一个线程,或者阻塞这些直到我知道前一个#任务完成。
目前它不使用Java Concurrency软件包,但我正在考虑更改以利用线程管理。
有谁有类似的解决方案或如何实现这一
我有一个将异步任务委托给线程池的进程。我需要确保某些任务按顺序执行。 因此,例如使用ExecutorService控制任务执行顺序
任务按顺序到达
任务A1,B1,C1,D1,E1,A2,A3,B2,F1
任务可以以任何顺序,除非有一个被执行自然依赖,所以a1,a2,a3必须按照这个顺序处理,或者分配给同一个线程,或者阻塞这些直到我知道前一个#任务完成。
目前它不使用Java Concurrency软件包,但我正在考虑更改以利用线程管理。
有谁有类似的解决方案或如何实现这一
建议当您提交Runnable
或Callable
到ExecutorService
回报你收到Future
。让依赖于a1的线程通过a1的Future
并致电Future.get()
。这将阻塞,直到线程完成。
所以:
ExecutorService exec = Executor.newFixedThreadPool(5);
Runnable a1 = ...
final Future f1 = exec.submit(a1);
Runnable a2 = new Runnable() {
@Override
public void run() {
f1.get();
... // do stuff
}
}
exec.submit(a2);
等。
另一种选择是创建自己的执行程序,将其称为OrderedExecutor,并创建一个封装的ThreadPoolExecutor对象数组,每个内部执行程序有1个线程。然后您提供一个机制,选择内部对象之一,例如,你可以通过提供您的类的用户可以实现一个接口,这样做:
executor = new OrderedExecutor(10 /* pool size */, new OrderedExecutor.Chooser() { public int choose(Runnable runnable) { MyRunnable myRunnable = (MyRunnable)runnable; return myRunnable.someId(); }); executor.execute(new MyRunnable());
OrderedExecutor.execute(的实现),那么将使用选择器得到一个int,你用这个池大小来修改它,这就是你的索引到内部数组中。这个想法是“someId()”将为所有“a”返回相同的值,等等。
当我在过去完成这项工作时,我通常已经通过一个组件处理了顺序,然后提交可执行文件/可运行到Executor。
有点像。
完成服务是能够得到的任务,因为他们完成,而不是试图轮询一堆的好方法期货。但是,您可能希望保留一个Map<Future, TaskIdentifier>
,当通过完成服务安排任务时填充这个Map<Future, TaskIdentifier>
,以便当完成服务为您提供完整的Future时,您可以确定它是哪个TaskIdentifier
。
如果你发现自己处于一个任务仍在等待运行的状态,但没有任何事情正在运行,并且没有任何事情可以安排,那么你就有循环依赖问题。
我编写自己的执行程序,以确保任务具有相同密钥的任务排序。它使用具有相同密钥的订单任务的队列映射。每个键控任务使用相同的键执行下一个任务。
此解决方案不处理RejectedExecutionException或委托Executor的其他异常!所以委托Executor应该是“无限的”。
import java.util.HashMap;
import java.util.LinkedList;
import java.util.Map;
import java.util.Queue;
import java.util.concurrent.Executor;
/**
* This Executor warrants task ordering for tasks with same key (key have to implement hashCode and equal methods correctly).
*/
public class OrderingExecutor implements Executor{
private final Executor delegate;
private final Map<Object, Queue<Runnable>> keyedTasks = new HashMap<Object, Queue<Runnable>>();
public OrderingExecutor(Executor delegate){
this.delegate = delegate;
}
@Override
public void execute(Runnable task) {
// task without key can be executed immediately
delegate.execute(task);
}
public void execute(Runnable task, Object key) {
if (key == null){ // if key is null, execute without ordering
execute(task);
return;
}
boolean first;
Runnable wrappedTask;
synchronized (keyedTasks){
Queue<Runnable> dependencyQueue = keyedTasks.get(key);
first = (dependencyQueue == null);
if (dependencyQueue == null){
dependencyQueue = new LinkedList<Runnable>();
keyedTasks.put(key, dependencyQueue);
}
wrappedTask = wrap(task, dependencyQueue, key);
if (!first)
dependencyQueue.add(wrappedTask);
}
// execute method can block, call it outside synchronize block
if (first)
delegate.execute(wrappedTask);
}
private Runnable wrap(Runnable task, Queue<Runnable> dependencyQueue, Object key) {
return new OrderedTask(task, dependencyQueue, key);
}
class OrderedTask implements Runnable{
private final Queue<Runnable> dependencyQueue;
private final Runnable task;
private final Object key;
public OrderedTask(Runnable task, Queue<Runnable> dependencyQueue, Object key) {
this.task = task;
this.dependencyQueue = dependencyQueue;
this.key = key;
}
@Override
public void run() {
try{
task.run();
} finally {
Runnable nextTask = null;
synchronized (keyedTasks){
if (dependencyQueue.isEmpty()){
keyedTasks.remove(key);
}else{
nextTask = dependencyQueue.poll();
}
}
if (nextTask!=null)
delegate.execute(nextTask);
}
}
}
}
+1。感谢那。我会使用这种植入方式,但我真的不知道这是不是标记为问题的最终答案。 – 2014-05-26 13:44:50
在Habanero-Java library,存在可用于表达任务之间的依赖关系,并避免线程阻塞操作数据驱动任务的概念。在封面下,Habanero-Java库使用JDK的ForkJoinPool(即ExecutorService)。
例如,你的用例的任务A1,A2,A3,...可以表述如下:
HjFuture a1 = future(() -> { doA1(); return true; });
HjFuture a2 = futureAwait(a1,() -> { doA2(); return true; });
HjFuture a3 = futureAwait(a2,() -> { doA3(); return true; });
需要注意的是A1,A2和A3是类型HjFuture的对象只是引用并且可以在您的自定义数据结构中进行维护,以指定在运行时任务A2和A3进入时的依赖关系。
有一些tutorial slides available。 你可以找到更多的文件,如javadoc,API summary和primers。
您可以使用Executors.newSingleThreadExecutor(),但它将只使用一个线程来执行您的任务。另一个选择是使用CountDownLatch。这里有一个简单的例子:
public class Main2 {
public static void main(String[] args) throws InterruptedException {
final CountDownLatch cdl1 = new CountDownLatch(1);
final CountDownLatch cdl2 = new CountDownLatch(1);
final CountDownLatch cdl3 = new CountDownLatch(1);
List<Runnable> list = new ArrayList<Runnable>();
list.add(new Runnable() {
public void run() {
System.out.println("Task 1");
// inform that task 1 is finished
cdl1.countDown();
}
});
list.add(new Runnable() {
public void run() {
// wait until task 1 is finished
try {
cdl1.await();
} catch (InterruptedException e) {
e.printStackTrace();
}
System.out.println("Task 2");
// inform that task 2 is finished
cdl2.countDown();
}
});
list.add(new Runnable() {
public void run() {
// wait until task 2 is finished
try {
cdl2.await();
} catch (InterruptedException e) {
e.printStackTrace();
}
System.out.println("Task 3");
// inform that task 3 is finished
cdl3.countDown();
}
});
ExecutorService es = Executors.newFixedThreadPool(200);
for (int i = 0; i < 3; i++) {
es.submit(list.get(i));
}
es.shutdown();
es.awaitTermination(1, TimeUnit.MINUTES);
}
}
我为这个问题创建了一个OrderingExecutor。如果将同一个关键字传递给方法execute()和不同的可执行文件,那么使用相同关键字执行runnables的顺序将按照调用execute()的顺序执行,并且永远不会重叠。
import java.util.Arrays;
import java.util.Collection;
import java.util.Iterator;
import java.util.Queue;
import java.util.concurrent.ConcurrentHashMap;
import java.util.concurrent.ConcurrentLinkedQueue;
import java.util.concurrent.ConcurrentMap;
import java.util.concurrent.Executor;
/**
* Special executor which can order the tasks if a common key is given.
* Runnables submitted with non-null key will guaranteed to run in order for the same key.
*
*/
public class OrderedExecutor {
private static final Queue<Runnable> EMPTY_QUEUE = new QueueWithHashCodeAndEquals<Runnable>(
new ConcurrentLinkedQueue<Runnable>());
private ConcurrentMap<Object, Queue<Runnable>> taskMap = new ConcurrentHashMap<Object, Queue<Runnable>>();
private Executor delegate;
private volatile boolean stopped;
public OrderedExecutor(Executor delegate) {
this.delegate = delegate;
}
public void execute(Runnable runnable, Object key) {
if (stopped) {
return;
}
if (key == null) {
delegate.execute(runnable);
return;
}
Queue<Runnable> queueForKey = taskMap.computeIfPresent(key, (k, v) -> {
v.add(runnable);
return v;
});
if (queueForKey == null) {
// There was no running task with this key
Queue<Runnable> newQ = new QueueWithHashCodeAndEquals<Runnable>(new ConcurrentLinkedQueue<Runnable>());
newQ.add(runnable);
// Use putIfAbsent because this execute() method can be called concurrently as well
queueForKey = taskMap.putIfAbsent(key, newQ);
if (queueForKey != null)
queueForKey.add(runnable);
delegate.execute(new InternalRunnable(key));
}
}
public void shutdown() {
stopped = true;
taskMap.clear();
}
/**
* Own Runnable used by OrderedExecutor.
* The runnable is associated with a specific key - the Queue<Runnable> for this
* key is polled.
* If the queue is empty, it tries to remove the queue from taskMap.
*
*/
private class InternalRunnable implements Runnable {
private Object key;
public InternalRunnable(Object key) {
this.key = key;
}
@Override
public void run() {
while (true) {
// There must be at least one task now
Runnable r = taskMap.get(key).poll();
while (r != null) {
r.run();
r = taskMap.get(key).poll();
}
// The queue emptied
// Remove from the map if and only if the queue is really empty
boolean removed = taskMap.remove(key, EMPTY_QUEUE);
if (removed) {
// The queue has been removed from the map,
// if a new task arrives with the same key, a new InternalRunnable
// will be created
break;
} // If the queue has not been removed from the map it means that someone put a task into it
// so we can safely continue the loop
}
}
}
/**
* Special Queue implementation, with equals() and hashCode() methods.
* By default, Java SE queues use identity equals() and default hashCode() methods.
* This implementation uses Arrays.equals(Queue::toArray()) and Arrays.hashCode(Queue::toArray()).
*
* @param <E> The type of elements in the queue.
*/
private static class QueueWithHashCodeAndEquals<E> implements Queue<E> {
private Queue<E> delegate;
public QueueWithHashCodeAndEquals(Queue<E> delegate) {
this.delegate = delegate;
}
public boolean add(E e) {
return delegate.add(e);
}
public boolean offer(E e) {
return delegate.offer(e);
}
public int size() {
return delegate.size();
}
public boolean isEmpty() {
return delegate.isEmpty();
}
public boolean contains(Object o) {
return delegate.contains(o);
}
public E remove() {
return delegate.remove();
}
public E poll() {
return delegate.poll();
}
public E element() {
return delegate.element();
}
public Iterator<E> iterator() {
return delegate.iterator();
}
public E peek() {
return delegate.peek();
}
public Object[] toArray() {
return delegate.toArray();
}
public <T> T[] toArray(T[] a) {
return delegate.toArray(a);
}
public boolean remove(Object o) {
return delegate.remove(o);
}
public boolean containsAll(Collection<?> c) {
return delegate.containsAll(c);
}
public boolean addAll(Collection<? extends E> c) {
return delegate.addAll(c);
}
public boolean removeAll(Collection<?> c) {
return delegate.removeAll(c);
}
public boolean retainAll(Collection<?> c) {
return delegate.retainAll(c);
}
public void clear() {
delegate.clear();
}
@Override
public boolean equals(Object obj) {
if (!(obj instanceof QueueWithHashCodeAndEquals)) {
return false;
}
QueueWithHashCodeAndEquals<?> other = (QueueWithHashCodeAndEquals<?>) obj;
return Arrays.equals(toArray(), other.toArray());
}
@Override
public int hashCode() {
return Arrays.hashCode(toArray());
}
}
}
我已经编写了我的赢了序列感知的执行器服务。它对包含某些相关参考和当前正在进行中的任务进行排序。
经过实施,我不认为这将有一个固定的线程池工作,因为线程可能()'一次和'f1.get所有块陷入僵局。 – finnw 2010-01-28 10:37:02
根据需要调整池的大小。 – cletus 2010-01-28 10:59:48
或使用缓存的线程池。 – finnw 2010-01-28 11:10:35