When looking at the User Actions dashlet you can select the particular action that you are interested in. A detailed tree of all the requests will be displayed in the bottom panel. In this section you can also find GET requests for js files. The response time column of this request will indicate how long it took to load the js file.
If your JS files are served from CDN you have such measure:
For 3rd party content (external domains)
If you have your JS as SubAction of main one that is related to some page load, this may work as well (you will have to play with rules for matching only JS loading.