Android平行（同步）图像下载

Question

伙计们帮忙请它很重要。

我需要在不到5分钟的时间内下载约2000张图片。所以我决定下载并行图像。

如果有人可以建议我更好地进行并行下载或告诉我我要出错的地方，我真的很感激。

在完成的任务的错误日志数总是各种各样的...

请帮助。

我使用异步任务执行由阿尔乔姆Zinnatullin

public class AsyncTaskExecutor { 

private static final int CORE_POOL_SIZE; 
private static final int MAXIMUM_POOL_SIZE; 
private static final int KEEP_ALIVE; 
private static final TimeUnit TIME_UNIT; 

private static final BlockingQueue<Runnable> concurrentPoolWorkQueue; 
private static final ThreadFactory concurrentThreadFactory; 
    private static final ThreadPoolExecutor concurrentExecutor; 

private AsyncTaskExecutor() {} 

static { 
CORE_POOL_SIZE = 5; 
MAXIMUM_POOL_SIZE = 128; 
KEEP_ALIVE = 1; 
TIME_UNIT = TimeUnit.SECONDS; 

concurrentPoolWorkQueue = new LinkedBlockingQueue<Runnable>(10); 
concurrentThreadFactory = new AsyncTaskThreadFactory(); 
concurrentExecutor = new ThreadPoolExecutor(CORE_POOL_SIZE,MAXIMUM_POOL_SIZE, KEEP_ALIVE, TIME_UNIT, concurrentPoolWorkQueue, concurrentThreadFactory); 
} 
/** 
* Concurrently executes AsyncTask on any Android version 
* @param task to execute 
* @param params for task 
* @return executing AsyncTask 
*/ 
@SuppressLint("NewApi") 
public static <Params, Progress, Result> AsyncTask<Params, Progress, Result> 
executeConcurrently(AsyncTask<Params, Progress, Result> task, 
Params... params) { 
if (Build.VERSION.SDK_INT >= Build.VERSION_CODES.HONEYCOMB) { 
task.executeOnExecutor(concurrentExecutor, params); 
} else { 
task.execute(params); 
} 
return task; 
} 

    /** 
* Thread factory for AsyncTaskExecutor 
* @author Artem Zinnatullin 
* 
*/ 
private static class AsyncTaskThreadFactory implements ThreadFactory { 
private final AtomicInteger count; 

{ 
count = new AtomicInteger(1); 
} 

@Override 
public Thread newThread(Runnable r) { 
return new Thread(r, "AsyncTask #" + count.getAndIncrement()); 
} 
} 

}

然后我使用循环下载我的所有图像。

for (PhotoToDBHelper photoToDBHelper : notDownloaded) { 
      try { 
       photoToDBHelper.download(m_context); 
      } catch (Exception e) { 
       // TODO: handle exception 
      } 

     } 

哪里下载方法是这样的：

public void download(Context context) throws Exception { 
    HttpClient httpClient = new DefaultHttpClient(); 
    HttpGet httpGet = new HttpGet(new URL(getPhotosUrl() + m_fileName).toString()); 
    HttpResponse response = httpClient.execute(httpGet); 
    HttpEntity entity = response.getEntity(); 

    InputStream inputStream = entity.getContent(); 
    filePath = Environment.getExternalStorageDirectory().toString() + getFoldeRelativePath() + m_fileName; 

    Utils.createFileOnExternalStorage(getFoldeRelativePath(), filePath); 
    FileOutputStream fos = new FileOutputStream(filePath, false); 

    byte[] buffer = new byte[20000]; 
    int byteRead = 0; 

    while ((byteRead = inputStream.read(buffer)) != -1) { 
     fos.write(buffer, 0, byteRead); 
    } 
    fos.close(); 
} 

当开始我的Android应用程序，我得到这个错误

02-25 01:08:30.133: E/AndroidRuntime(32357): FATAL EXCEPTION: main 
02-25 01:08:30.133: E/AndroidRuntime(32357): java.lang.RuntimeException: Unable to start activity ComponentInfo{"package".TabHostActivity}: java.util.concurrent.RejectedExecutionException: Task android.os.AsyncTask$3@2c0809b0 rejected from java.util.concurrent.ThreadPoolExecutor@2be18f48[Running, pool size = 128, active threads = 1, queued tasks = 10, completed tasks = 27] 
02-25 01:08:30.133: E/AndroidRuntime(32357): at android.app.ActivityThread.performLaunchActivity(ActivityThread.java:1967) 
02-25 01:08:30.133: E/AndroidRuntime(32357): at android.app.ActivityThread.handleLaunchActivity(ActivityThread.java:1992) 
02-25 01:08:30.133: E/AndroidRuntime(32357): at android.app.ActivityThread.access$600(ActivityThread.java:127) 
02-25 01:08:30.133: E/AndroidRuntime(32357): at android.app.ActivityThread$H.handleMessage(ActivityThread.java:1158) 
02-25 01:08:30.133: E/AndroidRuntime(32357): at android.os.Handler.dispatchMessage(Handler.java:99) 
02-25 01:08:30.133: E/AndroidRuntime(32357): at android.os.Looper.loop(Looper.java:137) 
02-25 01:08:30.133: E/AndroidRuntime(32357): at android.app.ActivityThread.main(ActivityThread.java:4441) 
02-25 01:08:30.133: E/AndroidRuntime(32357): at java.lang.reflect.Method.invokeNative(Native Method) 
02-25 01:08:30.133: E/AndroidRuntime(32357): at java.lang.reflect.Method.invoke(Method.java:511) 
02-25 01:08:30.133: E/AndroidRuntime(32357): at com.android.internal.os.ZygoteInit$MethodAndArgsCaller.run(ZygoteInit.java:784) 
02-25 01:08:30.133: E/AndroidRuntime(32357): at com.android.internal.os.ZygoteInit.main(ZygoteInit.java:551) 
02-25 01:08:30.133: E/AndroidRuntime(32357): at dalvik.system.NativeStart.main(Native Method) 
02-25 01:08:30.133: E/AndroidRuntime(32357): Caused by: java.util.concurrent.RejectedExecutionException: Task android.os.AsyncTask$3@2c0809b0 rejected from java.util.concurrent.ThreadPoolExecutor@2be18f48[Running, pool size = 128, active threads = 1, queued tasks = 10, completed tasks = 1032] 
02-25 01:08:30.133: E/AndroidRuntime(32357): at java.util.concurrent.ThreadPoolExecutor$AbortPolicy.rejectedExecution(ThreadPoolExecutor.java:1967) 
02-25 01:08:30.133: E/AndroidRuntime(32357): at java.util.concurrent.ThreadPoolExecutor.reject(ThreadPoolExecutor.java:782) 
02-25 01:08:30.133: E/AndroidRuntime(32357): at java.util.concurrent.ThreadPoolExecutor.execute(ThreadPoolExecutor.java:1303) 
02-25 01:08:30.133: E/AndroidRuntime(32357): at android.os.AsyncTask.executeOnExecutor(AsyncTask.java:564) 
02-25 01:08:30.133: E/AndroidRuntime(32357): at "package".data.AsyncTaskExecutor.executeConcurrently(AsyncTaskExecutor.java:58) 
02-25 01:08:30.133: E/AndroidRuntime(32357): at "package".TabHostActivity.checkUpdateState(TabHostActivity.java:130) 
02-25 01:08:30.133: E/AndroidRuntime(32357): at "package".TabHostActivity.onCreate(TabHostActivity.java:88) 
02-25 01:08:30.133: E/AndroidRuntime(32357): at android.app.Activity.performCreate(Activity.java:4465) 
02-25 01:08:30.133: E/AndroidRuntime(32357): at android.app.Instrumentation.callActivityOnCreate(Instrumentation.java:1049) 
02-25 01:08:30.133: E/AndroidRuntime(32357): at android.app.ActivityThread.performLaunchActivity(ActivityThread.java:1931) 
02-25 01:08:30.133: E/AndroidRuntime(32357): ... 11 more 

Answer 1

有一些需要解决的几个问题：

1. 2000张图片在5张分钟约为每秒7张图像。根据图像大小，网络带宽，处理器和存储速度等因素，这可能不适用于任何代码或并发量。吞吐量的数学公式为：每秒7 * N字节，其中N是平均图像大小。如果有大的图像，它可能是一个死角。
2. 该异常表示线程池已达到最大排队任务数（10）。这可以基于线程池管理器实现来配置。总的来说，试图同时启动2000个线程可能会加剧这种情况。建议队列中的排队任务（使用线程），监控它们的完成情况，然后开始更多，一次一个小组地说五（5）个小组。
3. 如果有任何服务器控制，请考虑使用精灵模型，其中多个图像存储在一个图像（文件/资源​​）中，由图像内的相对x/y坐标分开。这将使HTTP或任何网络请求的数量从2000减少到1（可能），从而显着提高性能。