Merge branch 'hotfix-1.4.2'

Author: arteymix

docs/customization.md

@@ -256,7 +256,7 @@ This feature can be disabled by setting the following configuration key to `fals
 rdp.settings.privacy.enable-anonymized-search-results=false
 ```
 
-## Customaxing the application's messages
+## Customizing the application's messages
 
 Some text displayed in RDP can be customized and/or internationalized. To do so, copy a provided `messages.properties` file in the working directory of the Web application and edit it. The file is found in
 [messages.properties](https://github.com/PavlidisLab/rgr/blob/master/src/main/resources/messages.properties)

docs/deployment.md

@@ -1,28 +1,36 @@
 # Deployment
 
-This section expands the process of deploying in a production environment. Make
-sure you are familiarized with the [installation](/installation) procedure
-first before attempting the described steps.
+This section expands the process of deploying in a production environment. Make sure you are familiarized with the
+[installation](/installation) procedure first before attempting the described steps.
 
 ## Web server
 
-There a few options you will likely want to specify when deploying your RDP
-instance.
+There a few options you will likely want to specify when deploying your RDP instance.
 
-* `-Dserver.port=<port>`: Port for the webserver to listen on.
-* `-Dspring.config.location=file:<faq location>`: Location to find the FAQ question & answers
-* `-Djava.security.egd=file:/dev/./urandom`:  Specify this if you receive logs such as:
-  _"Creation of SecureRandom instance for session ID generation using
-  [SHA1PRNG] took [235,853] milliseconds."_ The secure random calls may be
-  blocking as there is not enough entropy to feed them in `/dev/random`.
+### Server port
+
+Set `-Dserver.port=<port>` to adjust the port the embedded webserver is listening on.
+
+### Fast random number generation
+
+Specify `-Djava.security.egd=file:/dev/urandom` this if you receive logs such as: _"Creation of SecureRandom instance for
+session ID generation using [SHA1PRNG] took [235,853] milliseconds."_ The secure random calls may be blocking as there is
+not enough entropy to feed them in `/dev/random`.
+
+### Proxy configuration
+
+Set `-Dhttp.proxyHost` and `-Dhttp.proxyPort` to perform HTTP accesses through a proxy.
+
+Likewise, you can set up a proxy for FTP connections with `-Dftp.proxyHost` and `-Dftp.proxyPort`.
 
 ## General steps
 
 1. Create a working directory for the web service: `mkdir -p /project/directory`
 2. Move into that directory: `cd /project/directory`
-3. Download the latest release: `wget https://github.com/PavlidisLab/modinvreg/releases/download/vx.x/rdp-x.x.x.jar`
-4. Create `application.properties`, `faq.properties` and optionally `messages.properties`
-5. Test your setup: `java -jar rdp-x.x.x.jar`
+3. Download the latest release: `wget -O rdp-{{ config.extra.rdp_version }}.jar https://github.com/PavlidisLab/rdp/releases/download/v{{ config.extra.rdp_version }}/rdp-{{ config.extra.rdp_version }}.jar`
+4. Create `application.properties`, and optionally `faq.properties` and
+   `messages.properties` if you want to [customize messages](customization.md#customizing-the-applications-messages).
+5. Test your setup: `java -jar rdp-{{ config.extra.rdp_version }}.jar`
 6. Log into the database and activate other organisms and organ systems (optional)
 7. Deploy using Systemd (see below) or a Docker container
 
@@ -43,7 +51,7 @@ After=syslog.target
 User=tomcat
 Group=tomcat
 WorkingDirectory=/project/directory
-ExecStart=/bin/java -Xms256m -Xmx3g -Dserver.port=<port> -Djava.security.egd=file:/dev/./urandom -jar rdp-x.x.x.jar
+ExecStart=/bin/java -Xms256m -Xmx3g -Dserver.port=<port> -Djava.security.egd=file:/dev/urandom -jar rdp-{{ config.extra.rdp_version }}.jar
 SuccessExitStatus=143
 
 [Install]

docs/installation.md

@@ -5,15 +5,15 @@ This section describes the essential steps to deploy an RDP registry.
 ## Requirements
 
 - Java 8+
-- MySQL 5.5+ or equivalent
+- MySQL 5.7+ or equivalent
 - SMTP mail server
 
 ## Obtain a distribution of RDP
 
 Download the [latest jar distribution](https://github.com/PavlidisLab/rgr/releases/latest) from GitHub.
 
 ```bash
-wget -O rdp.jar https://github.com/PavlidisLab/rgr/releases/download/v1.3.2/rdp-1.3.2.jar
+wget https://github.com/PavlidisLab/rgr/releases/download/v{{ config.extra.rdp_version }}/rdp-{{ config.extra.rdp_version }}.jar
 ```
 
 The jar contains the core application, including an embedded webserver (Tomcat 8.5.x),
@@ -25,11 +25,15 @@ Create the database and an associated user that the application will use to
 store and retrieve data.
 
 ```sql
-create database <database name> character set utf8mb4 collate utf8mb4_unicode_ci;
+create database <database name> character set utf8mb4 collate utf8mb4_general_ci;
 create user  '<database username>'@'%' identified by '<database password>';
 grant all on rdp.* to '<database username>'@'%';
 ```
 
+If you're using MySQL 5.6 or prior, use the `utf8` character set and `utf8_general_ci` collate. It is a 3 bytes subset
+of the typical 4-bytes UTF-8 character encoding. Otherwise, you will face issues with the index size limit of 767 bytes
+due to some of our indexed columns containing 255 characters (4 * 255 = 1020 > 767, but 3 * 255 = 765).
+
 ## Setup application.properties
 
 In the working directory of the Web application, create an `application.properties`
@@ -60,7 +64,7 @@ That should be enough to get the Web service started. Now you can launch it by
 issuing the following command:
 
 ```bash
-java -jar rdp.jar
+java -jar rdp-{{ config.extra.rdp_version }}.jar
 ```
 
 If your email server is not properly configured, you will see an error from the

docs/release-notes.md

@@ -1,5 +1,28 @@
 # Release Notes
 
+## 1.4.2
+
+Restore rdp.site.shortname and rdp.site.fullname in `application.properties` since they're still being used in the user
+FAQ.
+
+Fix missing TaskExecutor when using an HTTP proxy. It now uses native`-Dhttp.proxyHost` and `-Dhttp.proxyPort` JVM flags;
+the proxy settings in `SiteSettings` and `InternationalSearchSetings` have been simply removed.
+
+Interpolate shortname and URLs in messages.
+
+Move all email text into messages.properties so they can be adjusted and translated
+
+Fix missing flash messages after confirming registration and contact email.
+
+Fix /api/users API endpoint for registries that are not using gene-level privacy. In this case, the gene privacy level
+is always set to `null` and the value in the profile must be used instead as a fallback.
+
+Require MySQL 5.7+ and add instructions for using MySQL 5.6 and prior. This is due to a index size limit that is busted
+with access and verification token when 4-bytes character encoding is used (i.e. utf8mb4).
+
+Fix incorrect primary key for researcher categories that resulted in the impossibility of adding more than one category to
+a given user profile.
+
 ## 1.4.1
 
 This patch release fix some issues encountered while running the 1.4.0 in a production setup.

mkdocs.yml

@@ -9,3 +9,7 @@ nav:
   - migration.md
   - api.md
   - release-notes.md
+plugins:
+  - macros
+extra:
+  rdp_version: 1.4.2

pom.xml

@@ -5,7 +5,7 @@
 
     <groupId>ubc.pavlab</groupId>
     <artifactId>rdp</artifactId>
-    <version>1.4.1</version>
+    <version>1.4.2</version>
 
     <developers>
         <developer>

src/main/java/ubc/pavlab/rdp/RemoteResourceConfig.java

@@ -3,30 +3,15 @@
 import lombok.extern.apachecommons.CommonsLog;
 import org.springframework.context.annotation.Bean;
 import org.springframework.context.annotation.Configuration;
-import org.springframework.http.client.SimpleClientHttpRequestFactory;
 import org.springframework.web.client.AsyncRestTemplate;
-import org.springframework.web.client.RestTemplate;
-import ubc.pavlab.rdp.settings.ApplicationSettings;
-
-import java.net.InetSocketAddress;
-import java.net.Proxy;
 
 @CommonsLog
 @Configuration
 public class RemoteResourceConfig {
 
     @Bean
-    public AsyncRestTemplate asyncRestTemplate( ApplicationSettings applicationSettings ) {
-        String proxyHost = applicationSettings.getIsearch().getHost();
-        Integer proxyPort = applicationSettings.getIsearch().getPort();
-        if ( proxyHost != null && proxyPort != null && !proxyHost.equals( "" ) ) {
-            SimpleClientHttpRequestFactory requestFactory = new SimpleClientHttpRequestFactory();
-            Proxy proxy = new Proxy( Proxy.Type.HTTP, new InetSocketAddress( proxyHost, proxyPort ) );
-            requestFactory.setProxy( proxy );
-            return new AsyncRestTemplate( requestFactory );
-        } else {
-            return new AsyncRestTemplate();
-        }
+    public AsyncRestTemplate asyncRestTemplate() {
+        return new AsyncRestTemplate();
     }
 
 }

src/main/java/ubc/pavlab/rdp/controllers/LoginController.java

@@ -25,6 +25,8 @@
 import ubc.pavlab.rdp.services.UserService;
 import ubc.pavlab.rdp.settings.ApplicationSettings;
 
+import java.util.Locale;
+
 /**
  * Created by mjacobson on 16/01/18.
  */
@@ -69,7 +71,8 @@ public ModelAndView registration() {
     @PostMapping("/registration")
     public ModelAndView createNewUser( @Validated(User.ValidationUserAccount.class) User user,
                                        BindingResult bindingResult,
-                                       RedirectAttributes redirectAttributes ) {
+                                       RedirectAttributes redirectAttributes,
+                                       Locale locale ) {
         ModelAndView modelAndView = new ModelAndView( "registration" );
         User userExists = userService.findUserByEmailNoAuth( user.getEmail() );
 
@@ -92,7 +95,7 @@ public ModelAndView createNewUser( @Validated(User.ValidationUserAccount.class)
         } else {
             user = userService.create( user );
             VerificationToken token = userService.createVerificationTokenForUser( user );
-            eventPublisher.publishEvent( new OnRegistrationCompleteEvent( user, token ) );
+            eventPublisher.publishEvent( new OnRegistrationCompleteEvent( user, token, locale ) );
             redirectAttributes.addFlashAttribute( "message", "Your user account was registered successfully. Please check your email for completing the completing the registration process." );
             modelAndView.setViewName( "redirect:/login" );
         }
@@ -107,7 +110,7 @@ public ModelAndView resendConfirmation() {
 
     @Transactional
     @PostMapping(value = "/resendConfirmation")
-    public ModelAndView resendConfirmation( @RequestParam("email") String email ) {
+    public ModelAndView resendConfirmation( @RequestParam("email") String email, Locale locale ) {
         ModelAndView modelAndView = new ModelAndView( "resendConfirmation" );
         User user = userService.findUserByEmailNoAuth( email );
 
@@ -123,19 +126,21 @@ public ModelAndView resendConfirmation( @RequestParam("email") String email ) {
             return modelAndView;
         } else {
             VerificationToken token = userService.createVerificationTokenForUser( user );
-            eventPublisher.publishEvent( new OnRegistrationCompleteEvent( user, token ) );
+            eventPublisher.publishEvent( new OnRegistrationCompleteEvent( user, token, locale ) );
             modelAndView.addObject( "message", "Confirmation email sent." );
         }
 
         return modelAndView;
     }
 
     @GetMapping(value = "/registrationConfirm")
-    public ModelAndView confirmRegistration( @RequestParam("token") String token ) {
+    public ModelAndView confirmRegistration( @RequestParam("token") String token,
+                                             RedirectAttributes redirectAttributes ) {
         ModelAndView modelAndView = new ModelAndView();
 
         try {
             userService.confirmVerificationToken( token );
+            redirectAttributes.addFlashAttribute( "message", "Your account has been enabled successfully, and you can now proceed to login." );
             modelAndView.setViewName( "redirect:/login" );
         } catch ( TokenException e ) {
             log.error( "Could not confirm registration token.", e );

src/main/java/ubc/pavlab/rdp/controllers/UserController.java

@@ -149,15 +149,16 @@ public ModelAndView support() {
     public ModelAndView supportPost( HttpServletRequest request,
                                      @RequestParam String name,
                                      @RequestParam String message,
-                                     @RequestParam(required = false) MultipartFile attachment ) {
+                                     @RequestParam(required = false) MultipartFile attachment,
+                                     Locale locale ) {
         ModelAndView modelAndView = new ModelAndView( "user/support" );
         User user = userService.findCurrentUser();
         modelAndView.addObject( "user", user );
 
         log.info( MessageFormat.format( "{0} is attempting to contact support.", user ) );
 
         try {
-            emailService.sendSupportMessage( message, name, user, request, attachment );
+            emailService.sendSupportMessage( message, name, user, request.getHeader( "User-Agent" ), attachment, locale );
             modelAndView.addObject( "message", "Sent. We will get back to you shortly." );
             modelAndView.addObject( "success", Boolean.TRUE );
         } catch ( MessagingException e ) {
@@ -222,13 +223,13 @@ public ModelAndView changePassword( @Valid PasswordChange passwordChange, Bindin
 
     @Transactional
     @PostMapping("/user/resend-contact-email-verification")
-    public Object resendContactEmailVerification( RedirectAttributes redirectAttributes ) {
+    public Object resendContactEmailVerification( RedirectAttributes redirectAttributes, Locale locale ) {
         User user = userService.findCurrentUser();
         if ( user.getProfile().isContactEmailVerified() ) {
             return ResponseEntity.badRequest().body( "Contact email is already verified." );
         }
         VerificationToken token = userService.createContactEmailVerificationTokenForUser( user );
-        eventPublisher.publishEvent( new OnContactEmailUpdateEvent( user, token ) );
+        eventPublisher.publishEvent( new OnContactEmailUpdateEvent( user, token, locale ) );
         redirectAttributes.addFlashAttribute( "message", MessageFormat.format( "We will send an email to {0} with a link to verify your contact email.", user.getProfile().getContactEmail() ) );
         return "redirect:/user/profile";
     }
@@ -237,7 +238,7 @@ public Object resendContactEmailVerification( RedirectAttributes redirectAttribu
     public String verifyContactEmail( @RequestParam String token, RedirectAttributes redirectAttributes ) {
         try {
             userService.confirmVerificationToken( token );
-            redirectAttributes.addFlashAttribute( "Your contact email has been successfully verified." );
+            redirectAttributes.addFlashAttribute( "message", "Your contact email has been successfully verified." );
         } catch ( TokenException e ) {
             log.error( MessageFormat.format( "{0} attempt to confirm verification token failed.", userService.findCurrentUser() ), e );
             redirectAttributes.addFlashAttribute( "message", e.getMessage() );
@@ -255,9 +256,9 @@ static class ProfileWithOrganUberonIds {
 
     @ResponseBody
     @PostMapping(value = "/user/profile", produces = MediaType.TEXT_PLAIN_VALUE)
-    public String saveProfile( @RequestBody ProfileWithOrganUberonIds profileWithOrganUberonIds ) {
+    public String saveProfile( @RequestBody ProfileWithOrganUberonIds profileWithOrganUberonIds, Locale locale ) {
         User user = userService.findCurrentUser();
-        userService.updateUserProfileAndPublicationsAndOrgans( user, profileWithOrganUberonIds.profile, profileWithOrganUberonIds.profile.getPublications(), profileWithOrganUberonIds.organUberonIds );
+        userService.updateUserProfileAndPublicationsAndOrgans( user, profileWithOrganUberonIds.profile, profileWithOrganUberonIds.profile.getPublications(), profileWithOrganUberonIds.organUberonIds, locale );
         return "Saved.";
     }
 

src/main/java/ubc/pavlab/rdp/events/OnContactEmailUpdateEvent.java

@@ -6,6 +6,8 @@
 import ubc.pavlab.rdp.model.User;
 import ubc.pavlab.rdp.model.VerificationToken;
 
+import java.util.Locale;
+
 @Getter
 @EqualsAndHashCode(callSuper = true)
 public class OnContactEmailUpdateEvent extends ApplicationEvent {
@@ -14,9 +16,12 @@ public class OnContactEmailUpdateEvent extends ApplicationEvent {
 
     private final VerificationToken token;
 
-    public OnContactEmailUpdateEvent( User user, VerificationToken token ) {
+    private final Locale locale;
+
+    public OnContactEmailUpdateEvent( User user, VerificationToken token, Locale locale ) {
         super( user );
         this.user = user;
         this.token = token;
+        this.locale = locale;
     }
 }

src/main/java/ubc/pavlab/rdp/events/OnRegistrationCompleteEvent.java

@@ -7,6 +7,8 @@
 import ubc.pavlab.rdp.model.User;
 import ubc.pavlab.rdp.model.VerificationToken;
 
+import java.util.Locale;
+
 /**
  * Created by mjacobson on 22/01/18.
  */
@@ -18,9 +20,12 @@ public class OnRegistrationCompleteEvent extends ApplicationEvent {
 
     private final VerificationToken token;
 
-    public OnRegistrationCompleteEvent( User user, VerificationToken token ) {
+    private final Locale locale;
+
+    public OnRegistrationCompleteEvent( User user, VerificationToken token, Locale locale ) {
         super( user );
         this.user = user;
         this.token = token;
+        this.locale = locale;
     }
 }

src/main/java/ubc/pavlab/rdp/listeners/UserListener.java

@@ -2,8 +2,6 @@
 
 import lombok.extern.apachecommons.CommonsLog;
 import org.springframework.beans.factory.annotation.Autowired;
-import org.springframework.context.event.EventListener;
-import org.springframework.http.HttpStatus;
 import org.springframework.stereotype.Component;
 import org.springframework.transaction.event.TransactionalEventListener;
 import ubc.pavlab.rdp.events.OnContactEmailUpdateEvent;
@@ -37,7 +35,7 @@ public void onRegistrationComplete( OnRegistrationCompleteEvent event ) {
         }
 
         try {
-            emailService.sendRegistrationMessage( event.getUser(), event.getToken() );
+            emailService.sendRegistrationMessage( event.getUser(), event.getToken(), event.getLocale() );
         } catch ( MessagingException e ) {
             log.error( MessageFormat.format( "Could not send registration email to {0}.", event.getUser() ), e );
         }
@@ -46,7 +44,7 @@ public void onRegistrationComplete( OnRegistrationCompleteEvent event ) {
     @TransactionalEventListener
     public void onContactEmailUpdate( OnContactEmailUpdateEvent event ) {
         try {
-            emailService.sendContactEmailVerificationMessage( event.getUser(), event.getToken() );
+            emailService.sendContactEmailVerificationMessage( event.getUser(), event.getToken(), event.getLocale() );
         } catch ( MessagingException e ) {
             log.error( MessageFormat.format( "Could not send contact email verification to {0}.", event.getUser() ), e );
         }

src/main/java/ubc/pavlab/rdp/model/Profile.java

@@ -84,7 +84,7 @@ public String getFullName() {
     private ResearcherPosition researcherPosition;
 
     @Enumerated(EnumType.STRING)
-    @Column(name = "researcher_category")
+    @Column(name = "researcher_category", nullable = false)
     @ElementCollection
     @CollectionTable(name = "user_researcher_category", joinColumns = { @JoinColumn(name = "user_id") })
     private final Set<ResearcherCategory> researcherCategories = new HashSet<>();