Clustering
Definition of a Cluster
A cluster is a parallel or distributed system consisting of independent computers that cooperate as a single system. In the case of the the In-Q-My Application Server Cluster, this group consists of two types of participants – dispatchers and servers. Generally speaking, clusters offer a way to utilize computer resources more productively in comparison to when the same numbers of machines are working standalone - the total result is greater than the sum of the separate parts.
Cost-efficiency is one aspect of choosing whether to set a cluster or to use
standalone machines. Another advantage of clusters is that, from technological point of view, they are more reliable, secure, productive, and deliver better overall service than standalone machines.
The goal of a cluster is to allow distributing of the computing load over the several machines participating in it. These machines communicate with the dispatchers and with each other in order to provide better – faster and more reliable - service to the client. The idea of clusters is that the machines in it represent one unit and clients perceive the cluster, as a single entity - not as a group, comprised of several computers. For clients the IP address of the dispatcher is the IP address of the whole cluster. Clients neither see the IP addresses of the servers in the cluster, nor have to know these multiple IP addresses. On the other hand, administrators know the IP addresses of the dispatchers and servers in the cluster.
When connected in a cluster, servers are still physically separated as different machines. The participants in the cluster share disk access and resources that manage data, but they do not share memory or processors.
Among the advantages of the clustering solution provided by the In-Q-My Application Server are scalability of the system, the ability to use machines, which are geographically dispersed, load balancing, fail-over, high availability and easy administration.
Scalability of the system
The ability to “plug-in” additional resources in the system at any time when more computing power is necessary is one of the main advantages of clusters and distributed systems.
Scalability allows to add new participants in the cluster when the load of clients’ requests is heavy. It can be done dynamically without having to stop the cluster. This feature gives flexibility and ability quickly to adapt to changing workloads and to provide adequate service to more users when this is necessary. In-Q-My Application Server supports up to 64 participants in the cluster, which allows to serve simultaneously millions of clients’ requests.
What is more important, because In-Q-My Application Server is J2EE™ compatible, it is possible the cluster to consist of machines with different hardware architectures and running under various OS – Windows™, UNIX™, Solaris, etc – which makes In-Q- My Application Server really cross-platform. This feature of J2EE™ compatible application servers allows building clusters that meet perfectly well the hardware requirements for more or less powerful, for cheaper or cutting-edge equipment.
Scalability is not only in regard to hardware. The architecture of In-Q-My Application Server gives users the ability to add new services to the already existing ones. This provides opportunities to easily integrate with existing external systems.
Globally Dispersed
In-Q-My Application Server allows the cluster to be globally distributed. The machines, connected in a cluster can belong to different WANs (Wide Area Network) and it is possible even to be distributed globally, i.e. in different countries on different continents. Global distribution of the participants in a cluster is a feature not supported by the most application servers, because their clustering technology uses simple IP multicast communication, which restricts, or at least firmly does not recommend, distributing the participants in a WAN.
Load Balancing
The idea behind load balancing is to send incoming requests for processing to the least busy server in the cluster. Load balancing increases productivity, because it allows to fully utilize the available resources and at the same time to deliver increased application throughput and optimal response time.
Generally speaking, load balancing for a group of servers can be done via hardware or software. Hardware load balancing usually provides less flexibility to the administrator, while software load balancing allows him or her more precise techniques for configuring the system and measuring performance.
The algorithm for load balancing developed by In-Q-My is as follows. At regular intervals, set by the administrator, the servers in the cluster send information to the dispatcher(s) about the level of their resource usage in percentage. When a new request arrives, the dispatcher(s) send it to the least loaded server. The algorithm takes into account not only the percentage of used resources, but also the percentage of free resources, which means that if an extremely big number of requests arrives simultaneously, not all of them will be send only to that server, which is the least busy at the moment, but will be directed to several machines in a way that will evenly distribute the load among them.
This algorithm is simple, yet powerful and functions faultlessly in clusters where servers run on different platforms and/or have different processing capabilities (not equal processing power or not equal amount of memory) as well as in homogeneous clusters where all servers have equal processing power.
Fail-Over Recovery
Fail-over recovery means that if one of the participants in the cluster is temporary or permanently unavailable, for instance because of a hardware crash, its functions are undertaken by another participant. As a result, requests are automatically redirected to a working server in the cluster and users and applications are not aware of what is happening, because the system continues to process requests properly. Fail-over recovery applies not only to the servers in the cluster, but to dispatchers as well. If there are two or more dispatchers in the system and one of them fails, the incoming requests are redirected to the IP of another dispatcher.
Fail-over recovery is provided for data, too. Data is replicated (mirrored) on other participants of the cluster and this ensures that even in case of a crash there will be no information loss. Having not just the data, but its latest update reflecting its state after a transaction has been committed or rolled back on a server in the cluster, is vitally important and bearing this in mind In-Q-My has developed a distributed database which allows synchronization of the cached data distributed in the cluster. The fail-over algorithms are based on the communication protocols for access to the system which are developed by In-Q-My. The fail-over algorithms prevent a single point of failure that affects the whole system. This leads to improved performance and higher availability.
High Availability
One of the most irritating messages a user sees is “The server could be down or is not responding.“ The expectations of users are that the server is up and running twenty-four hours a day, seven days a week. An unavailable server is a disaster for e-business. The losses due to downtime cannot be measured only in terms of revenues or missed profit. They damage the image of the company and ruin the confidence in it.
High availability is bound with scalability and fail-over recovery. These are the two factors (besides the hardware parameters of the participants in the cluster, of course) that predominantly influence the availability of the system. High availability has several aspects. Even in cases when the number of participants in the cluster is very big, and even when fail-over recovery and load balance algorithms are perfect, it is possible in theory billions of requests to arrive simultaneously. This poses risks for overloading of the system. In a situation like the above mentioned, it is more important first to finish processing of already received requests, than to receive new ones. The temporary denial to receive new requests for processing is not a high availability trade-off; it is simply provided to guarantee that under extreme circumstances the system will still continue to function without compromising security and the reliability of processed information.
Easy Administration
One of the greatest advantages of clusters is that they are easily administered. Through the Visual and Console administration facilities, In-Q-My Application Server allows integrated administering of the machines in the cluster instead of administering standalone servers. The presence of a “central point of command” leads to reduce costs for administration and training in terms of money, time and overall efficiency. Integrated administration creates a complete view of all the critical components and gives better control over total performance. It also allows fine-tuning of particular components/nodes on one or more of the participants in the cluster, which is a prerequisite for an adequate reaction in critical situations.
In-Q-My Application Server provides means for remote administration of the cluster, too. The administration tools allow secure monitoring both at start-up and during runtime. The runtime administration facilities give information about the status of execution of requests.
The cluster-level log files are another instrument useful for administering a cluster. In-Q- My provide cluster-level log files, which allow to monitor what has happened in the various modules of the system and which give administrators the freedom to choose what kind of information to be logged, where to log it, and in how much detail.
Cluster Architecture
Cluster components
There are two types of cluster components that participate in In-Q-My Application
Server Cluster. As mentioned above, these two kinds of participants are dispatchers and servers.
The Participants in In-Q-My Application Server Cluster
Dispatchers
The function of the dispatcher(s) in the cluster is to receive requests from clients and distribute them to the servers in the cluster for processing. After requests are
processed, the results are returned back to the dispatcher and from there to the
client. The route passed by requests differs depending on their complexity and the resources they have to access. The general scheme is discussed in more detail in the chapter about the Route of the Requests.
What is important to note is that dispatchers do not process requests. One might
Wonder what their role is – only to forward requests to the servers and the responses back to the clients? In the hierarchy in clusters this role is crucial, especially in regard to load balancing and fail over.
If a considerable number of requests is expected, dispatchers must be more powerful machines in order to be able to handle the requests without becoming the bottle-neck in the system, especially in regard to the fact that each dispatcher communicates with each of the servers in the cluster, as is seen from the picture above. But this does not necessarily mean that the dispatcher(s) must be the most powerful machine(s) in the cluster. On the other hand, recommended (hardware) configurations for the cluster depend also on the complexity of applications that are to be deployed and how much processing is to be done, as it will be discussed next.
Servers
The role of servers is to process client requests. What is important to note is that
servers do not communicate directly with clients. Each of the servers in the cluster communicates with all other servers and dispatchers, as the picture shows. The fact that each server communicates with all other servers means that there is replication and synchronization of data, which leads to faster and more secure processing of the requests and executing the business logic.
The business logic of applications (EJBs, JSPs, Servlets, etc) is hosted on servers. In case of complex applications that require a lot of processing and/or communication, servers can become a bottle-neck for the system if their number or processing power is insufficient.
Cluster Configurations
The cluster solutions, which In-Q-My Application Server provides are several. Depending on the goals set for a particular system – complexity of applications, expected number of requests, available hardware, etc - its administrator should choose how to configure the cluster. There is not a general recommendation about the number of the participants in the cluster, the ratio between dispatchers and servers, what services and components to deploy on which machines, etc. The main guiding light is the overall performance and resource usage of the system. For instance, when the workload is not heavy, it is not reasonable to have many dispatchers and/or many servers, because in this case the percentage of their resource usage will be low and the system will not be efficient.
In other cases the administrator(s) of the cluster need to configure a cluster comprised of as many machines as possible, because the requests are arriving "in bulk".
The recommended cluster configurations for application servers can be divided into two groups – with only one or with more than one dispatchers. Needless to say, the presence of at least one dispatcher and one server is necessary in order a cluster to be set. For very small systems this configuration - one dispatcher and one server - could be the right choice.
The first possible configuration is a single dispatcher – n servers. As stated above, the minimal possible number of servers in the cluster is one. This configuration allows building a stable and reliable system, but provides no fail-over and load balancing (obviously, if the only server crashes, there is nowhere else to redirect the requests to), which are among the main advantages of clustering and large-scale distributed systems.
More common are configurations, which involve two, three or more servers. Again, all client requests are received by the dispatcher, but in this configuration the dispatcher's role is to redirect the requests to the least busy server, where they are processed. This provides better fail-over, load balancing, and scalability, which leads to higher availability and better overall performance. The resources of the system are used better, because of the concurrency in processing of requests. The administration tools, provided by In-Q-My Application Server, make it an easy job to manage such a cluster configuration.
The "one dispatcher-many servers" configuration is suitable in cases when the number of arriving requests is medium, which allows all of them to be accepted by the dispatcher without turning it into the bottle-neck of the system. Still, the only dispatcher is a potential single point of failure. It is recommended for this configuration the applications and services to be homogeneously distributed over the servers in the cluster, because this allows better scalability.
The second possible configuration is m dispatchers - n servers (m < n). In this configuration it is not mandatory all dispatchers and servers to be on the same LAN (Local Area Network). They can belong to different WANs (Wide Area Network) and it is possible even to be distributed globally, i.e. in different countries on different continents.
The m dispatchers - n servers configuration unleashes the full potential of the clustering feature of In-Q-My Application Server, because it allows more requests to be received and processed. Again, there is no general recommendation about the ratio between the dispatchers and the servers in the cluster, nor about how to
distribute the services and the applications on the machines in the cluster.
Often it is better to distribute homogeneously those applications and services, which allow it. This affords real scalability and fail-over – the risks for a single point of failure are reduced further. In terms of scalability the improvement is clear - a new participant, no matter if a server or a dispatcher, can be plugged into the system at any time. It has only to register with the group and joins the cluster.
One of the issues that need to be considered in this configuration is overall performance. "By default" the overall performance of an m dispatchers – n dispatchers cluster improves in comparison to the other possible configurations. But it requires proper installation and configuration of the system, proper development of the applications, proper deployment. Another potential drawback is out of the reach of In-Q-My Application Server and the reason is the operation system(s), under which the cluster runs. It is obvious that not all operation systems are equally good as server platforms, although In-Q-My Application Server is a cross-platform server and this fact provides more flexibility in choosing the operation system(s) to run the cluster on.
Logical Modules in the Cluster
A cluster can be discussed in many aspects. One of them – the role of the participants in it – was the subject of the previous section. Another approach is to
look at the modules or the separate parts that integrate the pieces of code into a
system.
There are three types of logical modules in the structure of In-Q-My Application
Server:
· core systems (managers);
· core services;
· (additional) services.
These three types of logical modules build every server and dispatcher in the cluster.The list of core systems, core services and (additional) services differs depending on the type of the participant (a server or a dispatcher). The picture below illustrates the modules on a separate machine and the data stream of communication among them. Communication is bi-directional, as the red and blue arrows identify it.
Logical Modules
Managers
Managers are the very base of the servers and dispatchers in the cluster. Core systems are located on each of the machines in the cluster and are not distributed.Managers are the first to be started, when a server/dispatcher starts.
The managers included in In-Q-My Application Server are:
· Framework. This is the manager that initializes the rest of the managers. In a
way, it is a container for the managers.
· Managers, which have special functions in regard to clusters – Cluster
Manager and Dispatcher (respectively Server) Service Manager.
· Log Manager, Thread Manager, Ports Manager, Timeout Manager, Memory
Manager,Connections Manager, Classloader Manager. Their roles are explained in more detail in the technical documentation of In-Q-My Application
Server.
Different managers perform different tasks. The two managers that are (most) related to the cluster are Cluster Manager and Dispatcher/Server Service Manager.
As its name implies, Cluster Manager is directly related to the cluster. This is the
module in In-Q-My Application Server, which starts the cluster. If Cluster Manager on the first dispatcher node does not start, the whole cluster will not start. If Cluster Manager on a server node does not start, this element will not the join the cluster. The property files of Cluster Manager allow to set its properties.
Dispatcher Service Manager runs only on dispatchers and Server Service Manager runs only on servers. None of the managers on one machine communicates with its counterpart (the same manager) on another machine, no matter if a server or a dispatcher, not even Dispatcher Service Manager or Server Service Manager, besides Cluster Manager. All the communication in the cluster passes through the Cluster Managers of the servers and dispatchers, as it is shown in the picture on next page.
Core Services
Core Services are the second module in the kernel of the server. They provide the basic functionality of the server/dispatcher. On each of the machines in the cluster, Core Services are started by Service Manager, as seen from the picture above. Core Services in In-Q-My Application Server are: AI Service, SSL Service, Log, Deploy, Httptunneling, Crypt, P4, etc. The list of Core Services on servers and dispatchers is different. For instance, Admin, SecurityManager, DBMS, etc are Core Services on servers but quite logically are not run on dispatchers at all, because the role of dispatchers in clusters does not suppose the dispatcher to manage users or to take care of security, for instance.
(Additional) Services
Services are these logical modules in the architecture of the cluster, which extend its functionality. They are not part of the kernel of the server and the system could work properly without them, but the specified service will not be provided to clients. Again, the list of additional services on server and dispatcher nodes differs. It differs for the different servers and dispatchers (Server1, Server 2, etc.), too, because it reflects the role of the particular participant. Some of the additional services are: telnet, http, ejbentity, ejbsession, servlet_jsp, dbpool. It is up to the administrator to choose which services to start on which machine and this decision is made depending on the needs of the system (for instance what kind of applications are to be deployed in the cluster and on a particular machine).
Functionality and configuration of (additional) services is discussed in more detail in the documentation of In-Q-My Application Server.
The Stream Of Communication In The Cluster
This picture presents the logical modules in the architecture of In-Q-My Application Server and how they communicate with each other. For clarity, only the communication between the different participants in the cluster is presented here. The data stream of communication between the logical modules for each of the participants is given in the previous picture. It does not change when the machines are connected in a cluster.
The Route of Requests
Application servers and multi-tier architecture
It is easier to understand the logic of the route of processing requests by In-Q-My Application Server, if this process is discussed in connection with the role of Web application servers and multi-tier architecture.
The role of Web application servers is to provide the platform for deploying and
running sophisticated Web applications in addition to serving static pages. The
occurrence of application servers' technology a couple of years ago led to the advent of multi-tier architecture, which was an answer to the limitations imposed by traditional two-tier (client-server) architectures.
In multi-tier architectures clients do not communicate directly with the database, but send their requests to the application server, which then makes the connection with the database. The next picture shows the general route of requests in multi-tier architectures and the role of In-Q-My Application Server:
Multi-Tier Architectures And The Place Of Web Application Servers In Them
The next picture presents in more detail what is happening to the request inside the application server.
The Route Of Requests
The aim of this picture is to present in a simplified manner what happens to requests from the moment they are sent by the client to the dispatcher till the results are delivered back to the client. When looking at this picture, one should bear in mind that:
a) The route of requests shows the longest possible route a request could travel.
It is not meant to imply that every request passes along the whole route. Actually, usually requests do not pass through all of these steps or they follow the order (1-10) but skip some of the steps and go directly to a next step. For instance, if it is a JSP request, there is no need to go to the bean container in order to compile a servlet and to return it to the client.
b) On deliberately, in the picture there is no cluster. The picture shows only one
server besides the dispatcher. The reason is visually to simplify the presentation of what kind of processing happens to a request in the system, no matter on which of the many servers in the cluster. If the picture were to present several servers, as the actual situation in clusters is, the picture would contain n JSP Engines, n Servlet Engines, n Bean Containers, etc, distributed on several servers. In this case it will be more difficult for the user to understand the idea. It is more important to show the route in general and the different stages a request passes, than in which container or engine on which server what is done in particular, especially in regard to the fact that this depends almost entirely on the way the cluster is configured. And above all, presenting the route as “distributed” among the servers might suggest that this is the recommended configuration – a JSP Engine on Server 1 accesses the Servlet Engine on Server 4, which in turn passes it to the EJB Container on Server 45, etc. Once again it should be stated, that there is no recommended configuration what to be deployed where – it is up to the administrator, and above all to the particular application.
c) The path for delivering the results of processing is along the same route but in
reverse order, i.e. step 10, 9, 8, etc. and because of that it is not presented separately.
1. The browser (in this case, but it can be another client application) sends
request to the dispatcher(s) of the cluster.
2. The dispatcher(s) receive the requests and depending on its type, direct
it to a server to be processed. In the case of a browser, which sends an HTTP request using the HyperText Transfer Protocol, the dispatcher sends it to a server on which the HTTP Service of In-Q-My Application Server, is running. If the expected number of requests for static HTML pages is considerable, at this stage it is possible to add an Apache Web server and to send the request to it. The idea is to leave all dynamic HTML pages, which require more serious processing before being delivered to the client, to be handled by In-Q-My Application Server.
3. A server receives the request. This stage allows distributing the request
among the servers in the cluster. The corresponding service of In-Q-My Application Server receives the request. Depending on the type of the request, it can be directed by the dispatcher not only to HTTP Service but also to EJB Session Service, EJB Entity Service, Deploy Service, etc.
4. This stage allows distributing the request among the servers in the cluster, too. If it is a HTML request and there is a JSP in it, which must be processed before returned to the client, the request is passed to the JSP engine.
5. The servlet engine compiles the JSP into a servlet. The servlet can
either be returned to the client, or if the servlet uses beans, it goes further along the route. This stage allows distributing the request among the servers in the cluster, too.
6. If the client is an application that doesn't need HTTP and processing of
HTML pages, JSPs and Servlets, on step 3 the request is sent to the corresponding EJB service of In-Q-My Application Server and steps 4 and 5 are skipped. This stage does not allow distributing the request among the servers in the cluster, i.e. a given session bean can be processed only on the server, where it was received. But for faster overall performance it is advisable session beans to be replicated in the cluster.
7. At this stages entity beans are processed. In the case of entity beans
with container managed persistence they can be distributed among the servers in the cluster. There is always one server, which is responsible for all the transactions with a given bean. EJBs with bean managed persistence make direct directions with the database through the DB Pool.
8. The role of the buffer (storage) is to establish connections with the pool, to lock connections, transactions, bean instances, etc. It is closely related to entity beans.
9. DB Pool – holds connections to the Database (10) that are to be used
only for the needs of In-Q-My Application Server. The DB Pool is located only on one machine and is not replicated in the cluster. The binding to the DB Pool is global and it is accessed via the Naming Service.
10. This could be any external database, a driver for which is supported by
In-Q-My Application Server – Oracle, SAP-DB, etc. This is the last stage in processing the request by an application server.
Among the factors for occurrence of bottlenecks are:
· not enough processing power of some or all of the machines in the cluster or
on clients' side
· improper configuration and administration of the machines in the cluster
· not carefully thought of development and deployment of the applications
running in the cluster.
As this list implies, the reasons for bottlenecks occurrence can be in any of the tiers of multi-tier architecture.
Internet
This source of trouble has nothing to do with In-Q-My Application Server clustering solution but since it also deteriorates the overall efficiency of the system, it must be mentioned.
applications. One of the possible solutions is upgrading the bandwidth, and the other is again modifying the applications and components in a way, which will make them thinner. Another possible solution is mirrowing (even geographic) the servers in the cluster.
Bottlenecks In Dispatchers And Servers
Basically, the dispatchers and servers in the cluster could become a bottleneck, if
they are not properly configured for the number of requests and for the complexity of the applications that are running in the cluster. One possible reason could be the inadequate ratio between servers and dispatchers – either the number of dispatchers is not enough for the number of requests, or the servers (their total number or the number of servers, configured for particular services) are not enough and are flooded by the requests. The situation gets worse if the requests require more complex processing before being returned to the client.
The above-mentioned problem is more serious in clusters, which use hardware for load balancing, because usually hardware does not allow such a fine-tuning of the system. Some competitors’ clusters, which use load balancing hardware in place of a software dispatcher, do not provide load balancing and failover for clustered servlets and JSPs. The presence of dispatchers in In-Q-My Application Server cluster solution provides more flexibility in configuring the system and tools for administering it, too.
Sometimes the number of requests turns dispatcher(s) into a bottle-neck due to
external factors. Not all OS are equally good as platforms for Web application
servers. Under certain circumstances the operational system the dispatcher is
running on could impose limits on the number of requests, which can pass through a socket.
No comments:
Post a Comment
Hi,
Thanks for stopping by leaving a comment.
I am looking forward to make some new posts of such a kind. make sure to visit and leave comments which will help me in improving my blog. and don't forget to share with your friends.
by,
www.ieducated.blogspot.com
Regards sri vastav.