Start with apache httpd module development...

Apache by name is the start of a giant in http server which is more than 50% of the web world. currently known as httpd, its on such useful because of its popularization architecture where anyone can do there own changes by just  mounting a module.

So, today i'm going to demonstrate how to start with apache httpd dynamic module development with a hello world sample.

This tutorial is done on linux Mint destro. But any linux version will be no problem. Developing apache modules for windows is pretty straight forward, but need to prepare some tools which is not directly downloadable. The tool used is c/c++ compiler and apxs[APache eXtenSion] tool. apxs for windows is a purl module, and it need to have some configurations. But there wont be any change on application source of on development procedure. For the moment i'll continue for linux and will hope to continue windows instructions later.

Required components:
Apache httpd server[remember its not tomcat]
Apache Extension installed
gcc/g++ compiler
root access on mounting module to httpd and for restarting server

Steps
1.install apache2 and verify its running...

$ sudo apt-get install apache2

2.install apache extension tool


sudo apt-get install apache2-threaded-dev 

this will install required apache libs and includes. apache portable runtime[apr] may be install with apache server installation. so no need to bother on those.


3.Start development now


apache portable runtime is a world class c/c++ library for save coding  on a portable platform with light weight, but high performance. apache httpd server is the main project which was done using apr. so all the developments we are doing here is on top of apr. we only use major c/c++ libraries here.


If you have experience on linux kernel development, as you know there is a separate/dedicated set of libraries/include intended for kernel developments. Ideal situations here as well. instead of printf on std , we used printk in kernel modules. Here we use ap_rprintf. apache httpd modules are also very similar on functionality to linux kernel modules. You'll get to know those greatly.


There are two necessary includes in the module



#include "httpd.h" 
#include "http_config.h"

apache module is identified by httpd with a well defined data structure: AP_MODULE_DECLARE_DATA

there are major difference from apache 1.3x to 2.xx versions of httpd. So please be alert on which version you are using.here i'm doing with the latest[updated for version 1.4 on Feb 2012].

module AP_MODULE_DECLARE_DATA foo_module = {
    STANDARD20_MODULE_STUFF,
    foo_config_perdir_create,   /* create per-dir    config structures */
    foo_config_perdir_merge,    /* merge  per-dir    config structures */
    foo_config_server_create,   /* create per-server config structures */
    foo_config_server_merge,    /* merge  per-server config structures */
    foo_config_cmds,            /* table of configuration directives   */
    foo_register_hooks          /* register hooks */
};


The most important of here is the "register hooks"  other config's are kept NULL on this sample.
This hook is the join between httpd server and the module. This hook is created on server start. Actually speaking, this is a fork pipe name or a socket pipe in deep.

All requests to the httpd server will be mirrored to this hook and its the responsible of this hook to use a handler to execute the http request. So, lets see how we achieve this.







module AP_MODULE_DECLARE_DATA apache_test_module = 
{ 
        STANDARD20_MODULE_STUFF, 
        NULL, /* per-directory config creator */ 
        NULL, /* directory config merger */ 
        NULL, /* server config creator */ 
        NULL, /* server config merger */ 
        NULL, /* command table */ 
        apache_test_register_hook, /* request processing*/ 
}; 


This hook has a  definition of
static void 

apache_test_register_hook

(apr_pool_t *p) ;


What we have to do is to define a function to be handled by the requests coming from apache httpd pool.
Within this function lets define a separate handler function to look better coding.


static int ap_hook_http_req_handler(request_rec *r) ;


So, here is the actual hook function definition:


static void apache_test_register_hook(apr_pool_t *p) 
{ 
        ap_hook_handler( apache_test_handler, NULL, NULL, APR_HOOK_MIDDLE); 
}


What we have done now are, we have asked httpd to send us http sequests by sending a hook, and within the hook we have set a handler function to get the http request parameters.
Then what we have to do is to do the actual work we are all wanted to do from the start. To do something with the http requests. For example, lets say we want to send our own html tags to the browser from our module. What we have to do is to return the request with a proper html tag and with status 200k http response code. Now lets see how to achieve this.

static int apache_test_handler(request_rec *r) ;

so we define the actual handler function.

 #include "httpd.h"   
 #include "http_config.h"   
 static int apache_test_handler(request_rec *r)   
 {  
     if (strcmp(r->handler, "apache_test_module"))  
     {  
         return DECLINED;   
     }  
     ap_set_content_type(r, "text/html");   
     ap_rputs("<HTML>\n", r);  
     ap_rputs("\t<HEAD>\n", r);  
     ap_rputs("\t\t<TITLE>\n\t\t\tApache Test Module\n\t\t</TITLE>\n", r);  
     ap_rputs("\t</HEAD>\n\n", r);  
     ap_rputs("<H1>Hello Idiots....</H1>\n", r);  
     ap_rprintf(r, "Engineers are inside too..!!! <br>");  
     ap_rprintf(r, "<a href=\"http://blog.friendly.lk\">back to blog</a>\n");  
     ap_rputs("</BODY></HTML>\n" ,r);   
     return OK;   
 }   
 static void apache_test_register_hook(apr_pool_t *p)   
 {   
     ap_hook_handler(apache_test_handler, NULL, NULL, APR_HOOK_MIDDLE);   
 }   
 module AP_MODULE_DECLARE_DATA apache_test_module =   
 {   
     STANDARD20_MODULE_STUFF,   
     NULL, /* per-directory config creator */   
     NULL, /* directory config merger */   
     NULL, /* server config creator */   
     NULL, /* server config merger */   
     NULL, /* command table */   
     apache_test_register_hook, /* other request processing hooks */   
 };   

here at the first if() we check whether the request is from url http://localhost/apache_test_module otherwise we avoid processing.

so.. here is the final source save this in a file apache_test_module.c

and now its time to compile and install this module on apache server.

fist cd to the place where source apache_test_module.c is. then do the following with root privileged.

$ sudo apxs2 -ci apache_test_module.c 

Ok.. now the development part is done.But still, how the server will identify and forward requests to our module? for that, we need to configure our module in httpd.conf configuration file located at /etc/apache2/httpd.conf

 $ sudo gedit /etc/apache2/httpd.conf

add the following two two configurations on the fly.

LoadModule apache_test_module /usr/lib/apache2/modules/apache_test_module.so

SetHandler apache_test_module

the first tells httpd to load a new module called apache_test_module.so with the handler named apache_test_module.

the second says httpd request path handler to forwaard /apache_test_module URLs to above handler.

now its time to restart apache2 and check how it works...

$ sudo /etc/init.d/apache2 restart

then go to http://localhost/apache_test_module from your browser....

give me some comments on your results!!!!

Thanks!!!

On the way for my own thread pool [Semaphore] for multi-threaded applications

In Linux environment, there is a lack of ready made Database connection pool object or even a easily usable connection object invokees. So, i supposed to write my own implementations for the subject. But still there are some pretty good libraries to ease the implementations [eg: boost C++ lib]. But i thought to continue with standard way. Still this code is under development. The following code is still usable, but with lot of cavities. I'v implemented mutex mode here. But need to implement Semaphore  and avoid Blocked Waiting. That require to implement pThread. Your ideas are always welcome.

 /**get a new connection from the pool*/  
 sql::Connection* DBConnector::getConnection(void)  
 {  
  bool gotaconnection = false;  
  for(;!gotaconnection;)  
  {  
  sql::Connection* tmpConnection = NULL;  
  /*if lock is free, lock it and go inside*/  
  if(DBConnector::isFreeThenLock())  
  {  
   /*if there are free connections in the pool go and get it then free the lock and go*/  
   if(freeConnectionList->size() > 0)  
   {  
   tmpConnection = freeConnectionList->front();  
   this->freeConnectionList->pop_front();  
   this->usedConnectionList->push_back(tmpConnection);  
   TEST_LOGGER(SLOGGER_INFO_LOG_LEVEL, "GETCONNECTION : ", (char*)tmpConnection);  
   DBConnector::free();  
   gotaconnection = true;  
   return tmpConnection;  
   }else  
   {/*check if there can create more connections*/  
   TEST_LOGGER(SLOGGER_INFO_LOG_LEVEL, "CURRENT NUM OF CONNECTIONS : ", (freeConnectionList->size() + usedConnectionList->size()));  
   if((freeConnectionList->size() + usedConnectionList->size()) < (maxConnections))  
   {  
    bool successCreation = false;  
    for(;!successCreation;){  
    try{  
    TEST_LOGGER(SLOGGER_INFO_LOG_LEVEL, "CREATING MORE CONNECTIONS : ", "");  
     const std::string tmpUrl = std::string("tcp://").append(this->hostName).append(":").append(this->port);  
     tmpConnection = this->driver->connect(tmpUrl.c_str() , this->userName.c_str() , this->pswd.c_str());  
     tmpConnection->setSchema(this->schemaName.c_str());  
     tmpConnection->setAutoCommit(true);  
     this->usedConnectionList->push_back(tmpConnection);  
     TEST_LOGGER(SLOGGER_DEBUG_LOG_LEVEL, "CREATE CONNECTION : ", (char*)tmpConnection);  
     TEST_LOGGER(SLOGGER_INFO_LOG_LEVEL, "GETCONNECTION : ", (char*)tmpConnection);  
     successCreation = true;  
     gotaconnection = true;  
     DBConnector::free();  
     return tmpConnection;  
    }catch(sql::SQLException &e)  
    {  
     TEST_LOGGER(SLOGGER_FATAL_LOG_LEVEL, "ERROR RECREATING CONNECTIONS : ", e.what());  
     successCreation = false;  
     continue;  
    }  
    }  
   }else  
   {  
    TEST_LOGGER(SLOGGER_ERROR_LOG_LEVEL, "CAN'T ALLOCATE MORE CONNECTIONS, RETURNNING NULL : ", "");  
    DBConnector::free();  
    gotaconnection = true;  
    return NULL;  
   }  
   }  
  }  
  TEST_LOGGER(SLOGGER_ERROR_LOG_LEVEL, "COULD NOT GET A CONNECTIONS, WAIT AND RETRY: ", "");  
  DBConnector::free();  
  //Sleep(1);  
  }  
  return NULL;  
 }  
 Here is the sub functions that are on development to make ease the coding:  
 /**release a connection back to the pool*/  
 void DBConnector::releaseConnection(sql::Connection* connToRelease)  
 {  
  if(DBConnector::isFreeThenLock())  
  this->usedConnectionList->remove(connToRelease);  
  if(connToRelease!=NULL && !connToRelease->isClosed()){  
  this->freeConnectionList->push_back(connToRelease);  
  TEST_LOGGER(SLOGGER_INFO_LOG_LEVEL, "RELEASECONNECTION: ", (char*)connToRelease);  
  }else{  
  TEST_LOGGER(SLOGGER_FATAL_LOG_LEVEL, "DISCARD CONNECTION: ", (char*)connToRelease);  
  }  
  DBConnector::free();  
 }  
 /*check if the pool object is free and lock it its free*/  
 bool DBConnector::isFree(void)  
 {  
  return DBConnector::syncLock == 0 ? true : false;  
 }  
 /*check if the pool object is free to use for concurrent access*/  
 bool DBConnector::isFreeThenLock(void)  
 {  
  if(DBConnector::syncLock == 0)  
  {  
  DBConnector::syncLock = 1;  
  TEST_LOGGER(SLOGGER_INFO_LOG_LEVEL, "LOCKED: ", "");  
  return true;  
  }  
  return false;  
 }  
 /*release pool lock*/  
 bool DBConnector::free(void)  
 {  
  if(DBConnector::syncLock != 0)  
  {  
  TEST_LOGGER(SLOGGER_INFO_LOG_LEVEL, "FREED: ", "");  
  DBConnector::syncLock = 0;  
  return true;  
  }  
  DBConnector::syncLock = 1;  
  return false;  
 }  
 bool DBConnector::error(void)  
 {  
  DBConnector::syncLock = -1;  
  TEST_LOGGER(SLOGGER_ERROR_LOG_LEVEL, "ERROR: ", "");  
  return false;  
 }  

Useful Abstract Methods

This is somewhat older and useful set of code, but i can't remember where it was... may be when i was using ehCache for data pre-loading in says at MIT.

  /**  

    * Get Object Type  
    *  
    * @return  
    */  
   public Class getObjectType() {  
     return (this != null ? this.getClass() : CacheManagerEventListener.class);  
   }  

C# Simple drag and drop example

Sometimes back i wanted to create a XML design tool to development with some GUI based tool kit. But unfortunately i had no time to continue with the project.

I tried with some available code from internet to test the requirements..... just a copy past is here, till re-start the project sooner.

C# Simple drag and drop example:

// Form load event or a similar place private void Form_Load(object sender, EventArgs e)
{
    // Enable drag and drop for this form // (this can also be applied to any controls) this.AllowDrop = true;
    // Add event handlers for the drag & drop functionality this.DragEnter += new DragEventHandler(Form_DragEnter);
    this.DragDrop += new DragEventHandler(Form_DragDrop);
}
// This event occurs when the user drags over the form with // the mouse during a drag drop operation void Form_DragEnter(object sender, DragEventArgs e)
{
    // Check if the Dataformat of the data can be accepted // (we only accept file drops from Explorer, etc.) if (e.Data.GetDataPresent(DataFormats.FileDrop)) e.Effect = DragDropEffects.Copy;
    // Okay else e.Effect = DragDropEffects.None;
    // Unknown data, ignore it
}
// Occurs when the user releases the mouse over the drop target void Form_DragDrop(object sender, DragEventArgs e)
{
    // Extract the data from the DataObject-Container into a string list string[] FileList = (string[])e.Data.GetData(DataFormats.FileDrop, false);
    // Do something with the data...  // For example add all files into a simple label control: foreach (string File in FileList) this.label.Text += File + "\n";
}

SQL joins graphically simplified...

SQL[structured query language] is the language to get data from a DB. It let you retrieve data from a simple "select *" to many customized ways. "JOIN" is the keyword to retrieve data from multiple tables with a given relationship "ON".

There are few more keywords used with JOIN, and some are optional. ex: INNER, OUTER,LEFT,RIGHT,FULL,CROSS,... those keywords depends on the DB you are using.

As given early JOIN give us the opportunity to retrieve rows which are related on two tables. So those could be able to show in a Cartesian Venn diagram. So i have created two sample tables and added sample data to those. lets see how this can be simulated.

First lets create sample tables:


CREATE TABLE tblleft ( id_customer int(10), customer_name varchar(10) ) ENGINE=InnoDB DEFAULT CHARSET=latin1;
insert into tblleft (id_customer, customer_name) values (1, 'amara');
insert into tblleft (id_customer, customer_name) values (2, 'dasun');
insert into tblleft (id_customer, customer_name) values (3, 'saman');
insert into tblleft (id_customer, customer_name) values (4, 'nimal');
insert into tblleft (id_customer, customer_name) values (5, 'kasun');

CREATE TABLE tblright ( id_order int NOT NULL AUTO_INCREMENT, id_customer int, equipment varchar(10) DEFAULT '_', PRIMARY KEY (id_order) ) ENGINE=InnoDB DEFAULT CHARSET=latin1;

insert into tblright (id_order, id_customer, equipment) values (1, 1, 'food');

insert into tblright (id_order, id_customer, equipment) values (2, 2, 'car');

insert into tblright (id_order, id_customer, equipment) values (3, 6, 'computer');

insert into tblright (id_order, id_customer, equipment) values (4, 7, 'fan');

insert into tblright (id_order, id_customer, equipment) values (5, 3, 'mobile');

1:

First lets do the simple join and retrieve data which are common with id_customer in the two tables 

SELECT

    *

FROM

    tblleft l

JOIN tblright r

ON

    l.id_customer=r.id_customer;

hers is the results:

2:

SELECT

    *

FROM

    tblleft l

LEFT OUTER JOIN tblright r

ON

    l.id_customer=r.id_customer;

3:

SELECT

    *

FROM

    tblleft l

RIGHT OUTER JOIN tblright r

ON

    l.id_customer=r.id_customer;

4:

SELECT

    *

FROM

    tblleft l

LEFT OUTER JOIN tblright r

ON

    l.id_customer=r.id_customer

WHERE

    r.id_order IS NULL;

5:

SELECT

    *

FROM

    tblleft l

RIGHT OUTER JOIN tblright r

ON

    l.id_customer=r.id_customer

WHERE

    l.id_customer IS NULL;

6:

SELECT

    *

FROM

    tblleft l

LEFT OUTER JOIN tblright r

ON

    l.id_customer=r.id_customer

UNION

SELECT

    *

FROM

    tblleft l

RIGHT OUTER JOIN tblright r

ON

    l.id_customer=r.id_customer;

7:

SELECT

    *

FROM

    tblleft l

LEFT OUTER JOIN tblright r

ON

    l.id_customer=r.id_customer

WHERE

    r.id_order IS NULL

UNION

SELECT

    *

FROM

    tblleft l

RIGHT OUTER JOIN tblright r

ON

    l.id_customer=r.id_customer

WHERE

    l.id_customer IS NULL;

8:Cross join will provide the Cartesian product , but it cant be drawn by a Venn diagram.

SELECT

    *

FROM

    tblleft l

JOIN tblright r;

or 

SELECT

    *

FROM

    tblleft l ,

    tblright r;

I hope you get some idea on how those result sets are described with Venn diagrams.

Thank you!

Prevent XSS and request forgery and other common attack patterns | [: GaB :]

It looks like two-third of the attacks are based on three vectors:

1. SQL injection (25%)

$id="1;DROP TABLE users"; mysql_query("SELECT * FROM bars WHERE id=".$id); It is deeply shocking how many "developers" still don't get the message not to execute SQL commands forged from user input. Or at least, why are they still employed? This attack would be the most simple to prevent. You just always have to *escape* strings which are parameters of the sql query coming in as request parameters. If in doubt, what do I mean by that, simply escape *all* parameters of a query. Or better use queries parameterized as "SELECT ... WHERE id=? AND type=?"-s, your language must have a way to pass the values safely afterwards.
But stop, why in the world are these guys still writing any SQL queries in the first place?! Because script kiddies don't know what a persistence framework is. Stop handcrafting CRUDDAOs.

2. Cross-site scripting a.k.a. XSS (17%)

Rule of thumb: always check. If you use any output mechanism or view, check whether it escapes or not. If not, it is *your job* to do so! (Error messages printing also the invalid value are the simplest to overlook.)

* escapes by default.

* ${id} never escape by default. This is one of the most pithiest design decision in JSTL/EL. It is insecure by default. But of course you have the option to turn on security. Thank you very much. I understand the need for backward compatibility, but this is the kind of design problems which can be solved. Why not add an option to change the default behavior, and add fn:unEscapeXml() ?

<%@ taglib prefix="fn" uri="http://java.sun.com/jsp/jstl/functions" %> ${fn:escapeXml(id)} 

Plain EL expressions to produce output is pure evil.

* The output tags of your framework of choice is the most important to test. Never assume you already know it. In Spring 2.0 escaping in the form:input and form:errors tags were optional and off by default. In 2.5, it was switched to on by default in the documentation. But form:errors tag continued to print out unescaped output. In 3.0 everything seems to be fine by default in both.

Just to improve your chances always add this to your root context:

     defaultHtmlEscape     true  

3. Authentication and authorization (14%)

I guess mostly this happens by plain simply ignoring security logic. Most likely to happen when refactoring and ignoring to adopt security logic to the new business logic or domain structure. Those three should be independent by design. AOP or a security framework is a good candidate for separating the security concerns.
If you are not in control of the server side issues, use discovery tools like PhpSecInfo.

++1 Cross site request forgery a.k.a. CSRF and XSRF

Only 2% of the identified attacks fall into this category. But I also saw a lot of plain simply wrong attempts to fix this issue. This also shows how non-trivial it is for the average programmer to safeguard against this kind of attacks. It feels like many sites are vulnerable but not exploited yet.
To check the ingenuity of requests, first one must ensure, that GET requests are never changing state of the domain. (Trivial but not for everyone.)
Second, to always check POST requests' referer header values: they must match yours. This is sufficient for protection against XSRF, but has limitations.
Mostly suggested are tokens, a hidden field in every POST with a random variable identifying genuine requests (e.g. hash of the SESSIONID). The problem is, that every browser has a way to hack around it. You have to generate a new random token for each request to be used to fully safeguard yourself.
Alternatively you can change URLs dynamically (the token becomes part of the URL): for example you can use Spring's @PathVariable annotation bound to a @RequestParam("/action/{nextToken}").

Favorite Text circle on console

Here is a small code snippets for displaying famous charactor circle on the console.... you may enjoy it!!!

private String circle() {

        if ((System.currentTimeMillis() - startTime) < 100) {
            startTime = System.currentTimeMillis();
            return "\r" + current + "\t";
        }
      
        if (current == '|') {
            current = '/';
        } else if (current == '/') {
            current = '-';
        } else if (current == '-') {
            current = '\\';
        } else {
            current = '|';
        }
        return "\r" + current + "\t";
    }

deploying war file on Weblogic server by using ant scripts

Auto deploying war file on Weblogic server by using ant scripts

 server="t3://${weblogic.adminhost}:${weblogic.adminport}"  
 classpath="${weblogic.classpath}"  
 username="${weblogic.user}"  
 password="${weblogic.password}"  
 component="${app.name}:${app.name}"  
 debug="true";  
 classname="weblogic.ant.taskdefs.management.WLDeploy"  
 classpath="C:/bea/weblogic81/server/lib/weblogic.jar"  
 name="wldeploy";  
 debug="true" password="tropicAl"  
 source="${build}/test.war"  
 targets="fwmgb-ms01" user="system" verbose="true";   
 classpathref="weblogic.classpath" ;  
 user="${weblogic.user}" password="${weblogic.password}"  
 adminurl="t3://${weblogic.adminhost}:${weblogic.adminport}" source="dist/${app.name.war}"  
 action="deploy";  
 message="Successfully Deployed @http://${weblogic.adminhost}:${weblogic.adminport}/${app.name.war}"  
 adminurl="t3://${weblogic.adminhost}:${weblogic.adminport}" name="${app.name.war}"  
 action="undeploy" />  
 adminurl="t3://${weblogic.adminhost}:${weblogic.adminport}" name="${app.name.war}"  
 action="redeploy" />  
 c:\NewClearProject\MITDevelopmentDeployment\eclipseWorksPlace\MyWebApp>java w  
 eblogic.Deployer -verbose -noexit -name MyWebApp -adminurl t3://localhost:700  
 1 -user newclear -password mit12345 -undeploy  
 C:\Oracle\Middleware\wlserver_10.3\server\bin>java weblogic.Deployer -verbose -n  
 oexit -source C:\NewClearProject\MITDevelopmentDeployment\eclipseWorksPlace\NewC  
 learWeb\dist\MyWebApp.war -name MyWebApp -adminurl t3://localhost:7001 -us  
 er newclear -password mit12345 -deploy  
 weblogic.Deployer invoked with options: -verbose -noexit -source C:\NewClearPro  
 ject\MITDevelopmentDeployment\eclipseWorksPlace\MyWebApp\dist\MyWebApp.war  
  -name MyWebApp -adminurl t3://localhost:7001 -user newclear -deploy  

Lakdasun Trip Reports Archive » Kirigalpoththa Hike on 1st of January 2011

Lakdasun Trip Reports Archive » Kirigalpoththa Hike on 1st of January 2011

install Pulse- Audio Equalizer on Ubuntu 10.10

sudo add-apt-repository ppa:nilarimogard/webupd8
sudo apt-get update;
sudo apt-get install pulseaudio-equalizer