Part of the EllisLab Network
   
 
XML RPC > html in response > htmlspecialchars + javascript stripping
Posted: 25 May 2008 07:58 AM   [ Ignore ]  
Grad Student
Avatar
Rank
Total Posts:  88
Joined  08-01-2007

Hi there,

Building an XMLRPC server application.

The problem I am having is that I need to send html as a response, however the response is being recieved escaped using htmlspecialchars.

ie, this:

<html>
<
body>
This is some text
<
script
var 
thing="blah";
</
script&rt;
</
body>
</
html

becomes this in the response:

EDIT: Spaces put in artificially after ampersands to prevent the browser rendering them as entities.

<?xml version="1.0" encoding="UTF-8"?>
<methodResponse>
<
params>
<
param>
<
value>
<
struct>
<
member>
<
name>htmloutput</name>
<
value>
<
string>& lt;htmlgt;& lt;bodygt;This is some text.& lt;scriptgt; var thing=& quot;blahquot;;& lt;/scriptgt;& lt;/bodygt;& lt;/htmlgt;</string>
</
value>
</
member>
</
struct></value>
</
param>
</
params>
</
methodResponse

and once the client outputs the result, the script tag has also been stripped:

lt;htmlgt;& lt;bodygt;This is some text. var thing="blah";& lt;/bodygt;& lt;/htmlgt


My testing here is using the example lifted straight form the manual:


http://codeigniter.com/user_guide/libraries/xmlrpc.html

Except the server response is formulated as follows:

$string '<html><body>This is some text. var thing="blah";</body></html>';
        
        
$response = array (
                   array(
                         
'htmloutput' => array($string'string')
                        ),
                 
'struct'
                 
); 

Looking through the XML-RPC code I can see that htmlspecialchars is used, while not ideal I suppose I can get round that with htmlspecialchars_decode. Or alternativley alter the library so as to allow it to send html using cdata


I haven’t yet found how the tags are being removed, which is somewhere in the client library.

So my question is ...

How the hell do I send html with javascript?

Or shoudl I RTFM a bit more wink?
http://www.xmlrpc.com/spec

 Signature 


digital agency and web consultancy in london
http://www.cfp.co.uk

Profile
 
 
Posted: 25 May 2008 05:00 PM   [ Ignore ]   [ # 1 ]  
Grad Student
Avatar
Rank
Total Posts:  88
Joined  08-01-2007

k… looked further.

the xmlrpc class is cleaning the output using the CI input->xss_clean() method.

it happens within the xmlrpc->decode()
(called by send_request within the xmlrpc_client controller)

commenting a couple of lines solves the the script tag removing problem.

why is the data being xss cleaned here?

function decode($array=FALSE)
    
{
    
        $CI 
=& get_instance();

        if (
$array !== FALSE && is_array($array))
        
{
            
while (list($key) = each($array))
            
{
                
if (is_array($array[$key]))
                
{
                    $array[$key] 
$this->decode($array[$key]);
                
}
                
else
                
{
                    
//HDOTNET
                    //$array[$key] = $CI->input->xss_clean($array[$key]);
                
}
            }
            
            $result 
$array;
        
}
        
else
        
{
            $result 
$this->xmlrpc_decoder($this->val);
            
            if (
is_array($result))
            
{
                $result 
$this->decode($result);
            
}
            
else
            
{
                
//HDOTNET
                //$result = $CI->input->xss_clean($result);
            
}
        }
        
        
return $result;
    
 Signature 


digital agency and web consultancy in london
http://www.cfp.co.uk

Profile
 
 
Posted: 26 May 2008 04:19 AM   [ Ignore ]   [ # 2 ]  
Grad Student
Avatar
Rank
Total Posts:  88
Joined  08-01-2007

... well silence is deafening here, but suppose it is the weekend.

I’m not sure the default XMLRPC library is actually working as expected, so I may have uncovered a bug.

The manual states we input->xss_clean

.(JavaScript must be enabled to view this email address)

The XSS filter looks for commonly used techniques to trigger Javascript or other types of code that attempt to hijack cookies or do other malicious things. If anything disallowed is encountered it is rendered safe by converting the data to character entities.

Note: This function should only be used to deal with data upon submission. It’s not something that should be used for general runtime processing since it requires a fair amount of processing overhead.

also

If you want the filter to run automatically every time it encounters POST or COOKIE data you can enable it by opening your application/config/config.php file

$config[‘global_xss_filtering’] = FALSE in my config.php therefore I believe the XMLRPC class may not be behaving as it should, as it using the xss_clean function on the received data regardless of the config file setting.

Here’s how the decode method should look:

function decode($array=FALSE)
    
{
    
        $CI 
=& get_instance();

        if (
$array !== FALSE && is_array($array))
        
{
            
while (list($key) = each($array))
            
{
                
if (is_array($array[$key]))
                
{
                    $array[$key] 
$this->decode($array[$key]);
                
}
                
elseif($CI->config->item('global_xss_filtering') == TRUE)//HDOTNET
                
{
                    
                    $array[$key] 
$CI->input->xss_clean($array[$key]);
                
}
            }
            
            $result 
$array;
        
}
        
else
        
{
            $result 
$this->xmlrpc_decoder($this->val);
            
            if (
is_array($result))
            
{
                $result 
$this->decode($result);
            
}
            
elseif($CI->config->item('global_xss_filtering') == TRUE)//HDOTNET
            
{
                
                $result 
$CI->input->xss_clean($result);
            
}
        }
        
        
return $result;
    

would love it if someone could jump and tell me if I’m missing the point here.

 Signature 


digital agency and web consultancy in london
http://www.cfp.co.uk

Profile
 
 
Posted: 27 May 2008 10:03 AM   [ Ignore ]   [ # 3 ]  
Grad Student
Avatar
Rank
Total Posts:  88
Joined  08-01-2007

Wondering whether to post this as bug a or not.

The user guide implies that the XSS filter is not used by default, and must be turned on via the config file.

Yet the XML-RPC class is xss cleaning the data regardless of any setting in the config.

Anyone care to comment?

 Signature 


digital agency and web consultancy in london
http://www.cfp.co.uk

Profile
 
 
Posted: 22 June 2009 09:35 AM   [ Ignore ]   [ # 4 ]  
Grad Student
Rank
Total Posts:  61
Joined  10-31-2007

I am also having an issue where the xmlrpc xss cleaner is altering some html.  I am just going to modify the code.  I hope that this is fixed in the next version.7

Profile
 
 
Posted: 22 June 2009 11:06 AM   [ Ignore ]   [ # 5 ]  
Grad Student
Avatar
Rank
Total Posts:  88
Joined  08-01-2007

so maybe I am right….

 Signature 


digital agency and web consultancy in london
http://www.cfp.co.uk

Profile
 
 
Posted: 22 June 2009 11:53 AM   [ Ignore ]   [ # 6 ]  
Sr. Research Associate
Avatar
RankRankRankRankRank
Total Posts:  3153
Joined  06-11-2007

Don’t use XMLRPC, use REST. Much more fun to work with. grin

 Signature 

————————
Blog | Twitter | GitHub | BitBucket
————————-
PyroCMS - open source modular CMS built with CodeIgniter
PancakeApp - Simple, hosted invoicing/w project management

Profile
 
 
Posted: 22 June 2009 11:56 AM   [ Ignore ]   [ # 7 ]  
Grad Student
Rank
Total Posts:  61
Joined  10-31-2007

As for the html entities, that is not a bug.  HTML tags must be encoded before being sent so to not confuse with the xml, and therefore need to be decoded on return.

It is more an issue that I should be able to choose whether I want to xss clean everything that comes through XMLRPC, just like I can do with POST and GET.  I understand that one might want to remove any malicious code from a response, for example if one were to use XMLRPC as a web service backend to a public blog where you can’t have people posting [removed] tags in their blog entries to steal cookies, etc.  But in a controlled environment where you need the ability to send unaltered HTML, this is a major drawback.

I do not like the idea of editing the files in the /system/ directory since then I have to remember to change it when there is an update.

I am currently working on finding a nice clean way to encode/decode any strings that are sent or received so the XSS cleaner won’t even recognize them.

Profile
 
 
Posted: 29 June 2009 03:28 PM   [ Ignore ]   [ # 8 ]  
Grad Student
Rank
Total Posts:  61
Joined  10-31-2007

I have given up on trying to make some text encoding/decoding methods and simply commented out the lines that use the XSS cleaner and added one line.  For reference, in the file system/libraries/Xmlrpc.php:

Line 516:

// $array[$key] = $CI->input->xss_clean($array[$key]); 

Line 532:

// $result = $CI->input->xss_clean($result); 

Line 1130:

// $array[$key] = $CI->input->xss_clean($array[$key]); 

Line 1150:

$parameters[] $a_param;
// $parameters[] = $CI->input->xss_clean($a_param); 

Note, I am not using REST because I am an interacting with an interface in Flex.

Profile
 
 
Posted: 29 June 2009 04:45 PM   [ Ignore ]   [ # 9 ]  
Sr. Research Associate
Avatar
RankRankRankRankRank
Total Posts:  3153
Joined  06-11-2007

This REST client looks alright to me. It can be used at a basic level to just output XML and the useage is much easier, nothing to lose. grin

 Signature 

————————
Blog | Twitter | GitHub | BitBucket
————————-
PyroCMS - open source modular CMS built with CodeIgniter
PancakeApp - Simple, hosted invoicing/w project management

Profile
 
 
Posted: 29 June 2009 08:24 PM   [ Ignore ]   [ # 10 ]  
Grad Student
Rank
Total Posts:  61
Joined  10-31-2007

Hm.. interesting..  I will consider this if we decide to rewrite our protocol one day.  Thank you.

Profile