The return value inform Dynamo that if it needs to continue processing the rest of the page after the current handler is finished.
true - Normal processing of the remaining values continues, and the page specified by the form’s action attribute is served.
false - No further values are processed process after the handler is called, and the rest of the page is not served. For example, a handler that redirects the user to another page should return false
If it is a custom handler method, it is the developer responsibility to return appropriate value based on the above details.
Success and Failure URLs can be set as properties in your Formhandler and set the values using the Formhandler Component (.properties file). You may also set this value from the JSP as a hidden control but not recommended due to security.
From your handle method you may invoke checkFormRedirect(String pSuccessURL, String pFailureUrl, DynamoHttpServletRequest pRequest, DynamoHttpServletResponse pResponse)
You are correct that sending elements in the form handler is less secure, but there are a couple of things that help limit the security impact.
First is the session confirmation number which is, by default, required with a FormHandler form submission. This helps prevent some kinds of cross site scripting attacks in which a form rendered by someone else makes a post to your site via the customer's browser. Second is that newer versions of ATG do not allow off-site redirects without extra configuration (which helps with attacks that involve silently navigating away from your site to a site that can be used for phishing).
If you do decide to configure per-redirect-target FormHandlers, the newer $basedOn Nucleus property can help limit the amount of repeated configuration needed.
The value of hidden variables can be modified by using a Firebug or Google Chrome and submit that value to your server. After processing the form the form handler redirects to the URL that set in the success or error URLs, that you have accepted as hidden variables but the value has been changed by a user. At this point the server will redirect this to a 3rd party website specified and it can be potentially a malicious URL.
To avoid this ATG 10.1 has a servlet pipeline component called /atg/dynamo/servlet/pipeline/RedirectURLValidator. Set the enabled as true and you can define which are the outside hosts the system can redirect to (allowedHostNames property) .