Grails Ajax File Upload ProgressBar

Grails has the basic infrastructure in place to support AJAX-based file upload progress bars on your pages. In this article I will show you how to leverage the ExtJS Javascript library along with the Apache Commons File Upload library to achieve a nice user-friendly file upload progress bar in your Grails project. You can download a working Grails demo application at the end of this article in the Resources section.

Below is a screenshot of the demo application in action.


Background
File upload handling is, as everything else in Grails stacked on top of the Spring Framework. The Spring Framework in turn has delegated the job of managing streaming uploads to the Apache Commons File Upload library.
Commons File Upload takes care of very important server-side memory-management by allowing you to "stream" your uploads to disk instead of them being kept and handled entirely in memory. Read more about this at the Commons File Upload site.

In this article we will need to complete the following steps in order to achieve a working client-side progressbar:
  1. Implement a custom ProgressListener
  2. Subclass Spring's CommonsMultipartResolver
  3. Inject our custom ProgressListener and CommonsMultipartResolver classes into Spring via "conf/spring/resources.xml"
  4. Create an action in a controller which can be polled for upload progress info from client-side javascript
  5. Write appropriate client-side Javascript to poll the server for progress info

Prerequisites:
  • At the time of writing, Grails 1.0.1 ships with version 1.1.x of Apache commons-fileupload. Visit http://commons.apache.org/fileupload/ and get at least version 1.2
  • For the purposes of this tutorial I will be using ExtJS version 2.x (http://extjs.com/download), which comes with a nifty ready-built javascript progressbar.

Implement a custom ProgressListener
The commons file upload API allows us to register a class which implements the org.apache.commons.fileupload.ProgressListener interface. Once registered, this will allow us to receive progress updates as multipart data is being streamed to the server. Below is a code snippet from AjaxProgressListener.groovy (see Resources at bottom of article)

6:class AjaxProgressListener implements ProgressListener {
7:
8:
.
.
22: void update(long pBytesRead, long pContentLength, int pItems){
23:
24: session.setAttribute("progressMap", ["bytesRead": pBytesRead, "length" : pContentLength, "items" : pItems])
25: println("DEBUG UPLOAD: ${pBytesRead} : ${pContentLength} : ${pItems}")
26:
27: if (pBytesRead == pContentLength){
28: session.setAttribute("progressStatus", STATUS_DONE)
29: }
30:
31: }
32:}
As you can see in the above code snippet on line 24, we place the progress information into our session. We will need access to this information in our controller for retrieving progress information via javascript.

Subclassing Spring's MultipartResolver

Grails injects a "multipartResolver" property into our applicationContext. This property is actually an instance of Spring's CommonsMultipartResolver. CommonsMultipartResolver actually wraps the commons FileUpload class. However it does not expose the FileUpload functionality allowing us to register a progress listener.
In order to get our custom ProgressListener registered, we have to subclass CommonsMultipartResolver, and expose getter/setter methods for ProgressListener registration see lines 18 to 26 and our overridden factory method on line 47) in the
   1:package com.adeptiva.commons.upload
2:
3:import org.springframework.web.multipart.commons.CommonsMultipartResolver
.
.
11:import javax.servlet.http.HttpServletRequest
12:
13:class AjaxMultipartResolver extends CommonsMultipartResolver{
14:
15: private ProgressListener pListener
16: private HttpServletRequest request
17:
18: public void setProgressListener(ProgressListener p){
19: this.pListener = p
20: }
21:
22: public ProgressListener getProgressListener(){
23:
24: return this.pListener
25:
26: }
27:
28:
.
.
46:
47: protected FileUpload newFileUpload(FileItemFactory fileItemFactory) {
48:
49: FileUpload fu = super.newFileUpload(fileItemFactory)
50:
51: if (this.pListener != null){
52: this.pListener.session = this.request.session
53: fu.setProgressListener(this.pListener)
54: }
55:
56: this.pListener?.updateStatus(AjaxProgressListener.STATUS_UPLOADING)
57:
58: return fu
59: }
60:
61:}


Injecting our AjaxMultipartResolver into Grails/Spring
Now that we have our own implementation of the Spring CommonsMultipartResolver, it's time to take advantage of dependency injection to ensure that Grails uses our AjaxMultipartResolver. We modify our "conf/spring/resources.xml" as follows:
   1:<bean id="multipartResolver" class="com.adeptiva.commons.upload.AjaxMultipartResolver">
2: <property name="maxInMemorySize">
3:
4: <value>10240</value>
5: </property>
6: <property name="maxUploadSize">
7:
8: <value>1024000000</value>
9: </property>
10: <property name="progressListener">
11: <ref bean="progressListener"/>
12: </property>
13: </bean>
14:
15:<bean id="progressListener" class="com.adeptiva.commons.upload.AjaxProgressListener"/>
16:

Progress Information
The action which retrieves progress information is rather simple, and beyond doing some minor error checking is only responsible for constructing a JSON response (using builders of course).

   1:    def uploadInfo = {
2:
3: def progressMap = session.getAttribute("progressMap")
4: def progressStatus = session.getAttribute("progressStatus")
.
.
22: render(builder:'json'){
23: bytesRead(progressMap['bytesRead'] )
24: totalSize(progressMap['length'] )
25: status(progressStatus)
26: }
27: }

Our .gsp and Javascript
Between the .gsp and javascript files there remains only a few items to take care of:
  • Perform a form submission containing our file input via Javascript. This prevents the browser from navigating away from the current page after upload is complete.
  • Start an asynchronous task which polls the server for upload progress, updating our progressbar on each return.
The above is rather straightforward, and should be self-explanatory based on the supplied source code (js/application.js and views/home/index.gsp respectively) (see Resources section).

Resources:
You can download the full project demonstrating the above from HERE.

References:
File Uploading section from the Grails User Guide.
Apache Commons File Upload website


FIN.

22 comments:

dahernan said...

Good article, It could be fantastic view the progress bar in a plugin :D

jlorenzen said...

This is really neat.
I am also working on a file upload capability with extjx; unfortunately not with grails though.

I am curious to know what happens with your listener if multiple users are uploading files concurrently.

How does your update method determine which file you are being updated on?

Stephan February said...

jlorenzen:

As I'm tying progress information to session in the example, concurrent uploads from users should be fine.
However if the same user opens up another tab in their browser and starts uploading concurrently the results will probably be "undefined" :)

Stephan.

some guy said...

Hi, I just downloaded the app and ran it in grail 1.0.1, I get a missing property exception:

No such property: AjaxProgressListener for class: StorageController

Of course, this is supposedly injected right? I've looked to see if any configuration might need changing - but since I was hoping to find out how this worked, I am a little lost as to why it is broken :-)

some guy said...

OK, I understand. The file I was uploading was too small.

I tried a 100mb and I didn't get that exception


//if we don't have progress info in the session, it could
//indicate that the file upload was to small to require streaming
//and possibly finished before we could check progress
if (progressMap?.bytesRead == null){
render(builder:'json'){
bytesRead(1 )
totalSize(1 )
status(AjaxProgressListener.STATUS_DONE)
}
}


I think that is the gotcha.

OK, so if it is too small, then the file is still available, but check if listener is there?

Stephan February said...

@some guy: That's right. The Commons File Upload framework won't initialize the streaming upload for files that are below the configured threshold. Therefore we also won't have ProgressListener, and no progress info.

taverne said...

Hi,
Wonderfull grails script but :
it's possible to have some others informations about the upload like %. So we have the progress barre but can we have "12%" refresh for example
please.
best regards
thx
Julien Taverne

taverne said...

so if it's possible to have speed , time remaining, ... , i(d like too ;)

joshjdevl said...

So does spring automagically use streaming when the file size is greater than some threshold?

How and where is this configured?

joshjdevl said...

So when using the spring object MultipartFile, then thats when it uses the stream?

Dan said...
This post has been removed by the author.
Dan said...

I was implementing this example in a purely Java and Spring environment and ran into a null pointer exception in the newFileUpload method. It turns out that newFileUpload is called by the parent constructor and pListener hasn't been injected yet. After I looked around I found that the '?' means ignore null pointer errors. So for you Java guys, add line 56 to the if statement above it.

protected FileUpload newFileUpload( FileItemFactory fileItemFactory ) {
FileUpload fu = super.newFileUpload( fileItemFactory );

if( this.pListener != null ) {
this.pListener.setSession( this.request.getSession() );
fu.setProgressListener( this.pListener );
this.pListener.updateStatus( AjaxProgressListener.STATUS_UPLOADING );
}

return fu;
}

matthias said...

Very helpful post indeed.

I was wondering though: You store a reference to a servlet request, and you access that request without initializing it.

where is the code for setting the request to a valid object?

Stephan February said...

@matthias: CommonsMultipartResolver is initialized by the Spring Framework.

tcurdt said...

Nice! Unfortunately does not work 1.1 out of the box. Would also be good to leave the default HSQL db in by default.

cenkcivici said...

The listener is a singleton right? If so setting session on it has the potential risk of overwriting other concurrent upload information..

little-michi said...

I really like this article ;-)

I used the uploadBar in my grails-app, but since I upgraded to grails1.1 it's broken :(

I think it because of lazy fetching / session handling,trouble starts at line 53:
fu.setProgressListener(this.pListener)

I'm still searching for a workaround...

Michaela

little-michi said...

I found the error...

I forgot to delete the old commons-fileupload.jar from the grails lib folder...

oh that hurts ;)

michaela

Lars E said...

Is the source available at som other location? The link below does not work...

Stephan February said...

Hi Lars

I've fixed the links. You should be able to download the example project now.

Regards
Stephan

Lars E said...

@Stefan: thanks for this article and for the fixed link.

@cenkcivici: You are right, I experienced this myself. Setting the request and calling newFileUpload is not thread safe. I ran into conditions where the upload information was mixed up.

I solved this by instantiating a new pListener on each newFileUpload call.

Chris said...

This is a great article and works great. Except on my first try. When I do the first upload, I get the missing property for "AjaxProgressListener" in my controller. I'm using a 500mb file so I know its large enough. I'm also waiting 2 seconds before I even query the progress the first time. If I try to upload again then it works fine. It's just the first upload after the app starts. Maybe a session problem??? I'm on Grails 1.1.1

Post a Comment