Clarity Service Management

Solve export problem with utf8 characters in whereclause 

03-05-2019 08:42 AM

This document provides a suggestion to solve the following Problem:

If you want to Export search results from SDM through the Export functionality,

and using utf8 characters in your search criteria, the Export result

might be empty or incomplete.

 

For example , If I search for Contacts with Last Name like '%ü%', so I am looking for people like me (Michael Müller),

I find a lot of contacts , but if I want to export this result, it fails, the result is empty!

 

One reason for this behavior seems to be, that the underlaying whereclause , which includes utf8 characters, is transmitted as urlencoded,

but the PDMExport Servlet running in the SDM tomcat container interpretes this data as ISO-8859-1 encoded, which is wrong in this case.

 

The tomcat request encoding for the PDMExport servlet can be changed to UTF-8 by the following changes to

$NX_ROOT/bopcfg/www/CATALINA_BASE/webapps/CAisd/WEB-INF/web.xml

Add behind : <!-- Add filter here -->
    <filter>
        <filter-name>setCharacterEncodingFilter</filter-name>
        <filter-class>org.apache.catalina.filters.SetCharacterEncodingFilter</filter-class>
        <init-param>
            <param-name>encoding</param-name>
            <param-value>UTF-8</param-value>
        </init-param>
        <async-supported>true</async-supported>
    </filter>

and add behind : <!-- Add filter-mapping here -->
  
    <filter-mapping>
        <filter-name>setCharacterEncodingFilter</filter-name>
        <servlet-name>PDMExport</servlet-name>
    </filter-mapping>

 

restart tomcat by

pdm_tomcat_nxd -c STOP

pdm_tomcat_nxd -c START

 

Let me know, if you face the same problem,

and if this suggestion helps in your situation as well,

and if not, let me know as well.

 

Thanks and regards

......Michael 

Statistics
0 Favorited
3 Views
0 Files
0 Shares
0 Downloads

Tags and Keywords

Comments

03-07-2019 09:49 AM

I just recognized that 17.0 is working fine with the same scenario, as well as 17.1

So this seems to be a 17.1.0.1 problem only!

Regards

...............Michael

03-07-2019 03:18 AM

Hi Imran.

Great. Yes, I saw this yesterday as well.

The solution above handles this problem for all(?) previous releases / patchlevel.

I recognized that the fix in 17.1.0.2 and 17.2 uses the same "filter" approach as above.

But can you tell me, why development introduced a new class for the filter definition, instead using the standard tomcat one?

And why wasn't it handled as it is usual with a simple

request.setCharacterEncoding("UTF-8");

in the PDMExport servlet, as it is done in all other SDM servlets?

If so, you would have less overhead , less code to maintain, and less config to take care of.

Regards.

.............Michael

03-06-2019 10:53 PM

Hi Michael,

 

This problem is fixed in 17.2 and probably 17.1 RU#2 too.

 

Regards,

Imran

03-06-2019 01:18 PM

Great Michael! Thks

03-06-2019 03:18 AM

Just as an additional comment:

In my opinion, this is a bug in the PDMExport servlet, and should be fixed in the code!

The above suggestion is just a workaround.

Regards

...........Michael

Related Entries and Links

No Related Resource entered.