The great grandson of Husnu Sensoy

July 9, 2009

_pga_max_size = 200M : Maximum size of the PGA memory for one process ?!?

Filed under: Oracle — kocakahin @ 11:08 pm
Tags: , ,

For almost two years, I know a lot of  people like to play UNSUPPORTED Oracle parameters for managing sql workarea size policy of Oracle. Although Mr Lewis has published tens of illustrations proving that it is impossible to put general arguments about those parameters, people are insisting on playing with those parameters and hoping that their proposals on the nature of those parameters are valid.

In this post I will be discussing PGA_AGGREGATE_TARGET (”PGA_AGGREGATE_TARGET specifies the target aggregate PGA memory available to all server processes attached to the instance”) and  _PGA_MAX_SIZE (”Maximum size of the PGA memory for one process) parameters from the perspective of PL/SQL development by examplifying things. At the bottomline you will understand that _PGA_MAX_SIZE is not the maximum PGA memory for one process but it is the turn point for SQL workarea content to be purged on to a temporary tablespace. Moreover although PL/SQL local memory structures are also allocated within PGA, those parameters have nothing to do with the memory allocation of PL/SQL codes. They are for SQL workarea requirements like hash join, group by, sort, and direct path load buffers.

Filling up the World’s Simplest Collection

The machine I did those tests was a 4 CPU Sun Sparc machine with 64 GB memory and running Sun Solaris. The database version was 10.2.0.4 and PGA_AGGREGATE_TARGET parameter was 24M. All UNSUPPORTED parameters had their default values. So _PGA_MAX_SIZE was 200M for that instance. So once you read the definition you think that it is not allowed for a single Oracle process to exceed 200M of allocated PGA memory (or used PGA memory if you like). Look at the following demonstration:

declare
type myarray_t is table of varchar2(32767);
myarray myarray_t := myarray_t();
i       pls_integer := 1;
begin
dbms_application_info.set_module(’PGA USAGE TEST’,'SIMPLEST COLLECTION : 0′);
loop
myarray.extend;
myarray(i) := dbms_random.string(’u', 32767);
dbms_application_info.set_module(’PGA USAGE TEST’,'SIMPLEST COLLECTION : ‘||to_char(i));
i := i + 1;
end loop;
end;
/
declare
  type myarray_t is table of varchar2(4000);
  myarray myarray_t := myarray_t();
  i       pls_integer := 1;
begin

  dbms_application_info.set_module('PGA USAGE TEST','SIMPLEST COLLECTION : 0');

  loop
    myarray.extend;
    /**
    Longest string that dbms_random.string can generate is 4000 even if you set len parameter to higher values.
    */
    myarray(i) := dbms_random.string('u', 4000);
    dbms_application_info.set_module('PGA USAGE TEST','SIMPLEST COLLECTION : '||to_char(i));
    i := i + 1;
  end loop;
end;
/

While executing this script from one  session , monitor the PGA usage from v$process view periodically by using

set linesize 500

column display_value format a5

select * from v$sql_workarea_active;

select p.PGA_USED_MEM / 1024 / 1024 pga_used,

       p.PGA_ALLOC_MEM / 1024 / 1024 pga_alloc,

       p.PGA_FREEABLE_MEM / 1024 / 1024 pga_freeable,

       s.MODULE,

       s.ACTION

  from v$process p, v$session s

 where p.ADDR = s.PADDR

   and s.MODULE = 'PGA USAGE TEST'

 order by p.PGA_ALLOC_MEM desc;

If there is no other running task on the system, you won’t see any kind of workarea allocation by the server. So all those PGA parameters have nothing to do with your execution. And if you plot the all over PGA memory allocation for our execution you will see something similar to:

pga_usage_in_plsql

As you see the PGA usage of a single process can be much larger than magic 200M when it comes to PL/SQL. Those parameters have nothing to do with PL/SQL collections’ memory usage. More dramatically if you continue to extend the collection, at some point your OS starts to swap the Oracle memory onto disk. So this means an evil developer can lead in a system crash by just doing some careless memory allocation.

One final thing to be mentioned is the ratio between PGA_ALLOCATED and PGA_USED. I have noticed that as the size of collection increases in time, Oracle starts to converge to a fix ratio (in may case it was ~2.65) for PGA_ALLOCATED/PGA_USED. It is good in the sense that Oracle keeps a constant correlation between used PGA and allocated PGA for PL/SQL collections.

correlation

 Conclusion

Sorry but Oracle doesn’t promise for anything it has not documented. Although x$ksppi.ksppdesc column describes _PGA_MAX_SIZE  parameter as the maximum PGA allocated for an Oracle process, it doesn’t say anything specific about PL/SQL memory usage. So you should ensure that your code is not using TOO MUCH MEMORY. For example, you should guarantee that you are not trying to bulk fetch a very large table at a time in your code, or you are not assuming that  the table you are fetching will not grow in time.

June 25, 2009

APEX Roadmap : APEX 4.0

Filed under: Event, Oracle — kocakahin @ 9:51 am
Tags: , , , ,

Since 2006 I haven’t been developing by using Oracle APEX. As far as I remember version was 1.5 or 1.6 at that time. In time, I heard lots of great things about APEX from many people (maybe the most important one is Tonguc). So I have decided to attend Oracle Application Express 4.0 sessions at ODTUG this year. As Oracle guys have mentioned in their presentations, I will also stress that the things written in this post is just reflecting a direction. No promise that you will find all in the production release. So, here are the list of a few things thought to be available by APEX 4.0

Easy Way to Develope : Web Sheets

Even for simple form & report based applications (acutally it is a much general problem) the major problem is that users always ask for something different from what developers put on.It seems that Oracle APEX development will be possible by the actual users of the applications in the near future. With Web Sheets, Oracle’s aim is to let user to produce his/her own applications on APEX without bothering any developers. Builder and Run Time components of APEX are being merged within Web Sheets to let the development of dirty quick applications by users themselves.

For example, instead of a marketing guy sending a report to a sales guy in an Excel sheet (including some tabular data, charting, may be some input fields, etc in it) attached to an e-mail, he can simply put the report on intranet via an application easily developed on APEX. He or others can add some attachments, tags, comments on some data. Moreover, if the producer of the application or anyone else changes any piece of data, all chages will be audited. Even more they can add some notifications for their reports so that reports can be automatically sent to their e-mail boxes.

Obviously there are some differences in developing a classical APEX application and a Web Sheet. Here are just three differences between an APEX application and a Web Sheet:

APEX Application Web Sheets
Primary Key Maintenance Trigger or Sequence Automatically Managed
Page Flow Controlled by branches Limited
Look & Feel Themes & templates Basic Control

Declarative Way of Scripting : Dynamic Actions

One of the most popular things on web technologies is to put some intelligence on client-side by using cool AJAX stuff. The major problem in here is for such database geeks like me who doesn’t know anything about writing a piece of javascript. The basic idea behind dynamic actions is to allow developer to create client side AJAX actions in a declarative fashion without writing a piece of code.

Better Charts : Integration with AnyChart 5.1

When I gave up developing APEX, the only charting option you had was SVG (Support Vector Graphics). It seems that in time Oracle put another option of charting, namely flash charts into APEX. APEX uses the flash charts developed by an Oracle Partner company AnyChart. In version 4.0 the latest version of this charting module (currently 5.1) will be embedded into APEX.

Integration with Other Applications : REST Web Services

It seems that currently APEX does support SOAP Web Service calls from the applications you have built with APEX. By APEX 4.0, Oracle will also allow you to call REST Web services. Besides the architectural differences between a SOAP web service and a REST web service, the list of a few REST web services seems to be very attractive for many applications:

  • Yahoo Map
  • Flicker
  • Google Picasa
  • Amazon Products

And in the demo, Oracle guys show that how it is easy to embed a map into your application.

Extensibility : Plug-ins

Due to the fact that it is impossible /impractical to satisfy all individual customer needs, by APEX 4.0 Oracle introduces the concept of plug-in. You will be designing your own plug-ins by using bare PL/SQL APIs and you will be able to use it as an item in your applications and hopefully you will share that item for me (or the rest of the community) to use it. I think this is one of the best features and we will be using very fancy APEX plug-ins developed by many people just like the plug-ins developed for Firefox. In the presentation one classical example of these plugins was demonstrated. That was infamous star rating plugin

star rating plugin

Finally, I am pretty sure that there will be many more in APEX 4.0 when it becomes available. So hope to return back to development some day and meet with APEX 10.0 at that time J

June 21, 2009

Before ODTUG Kaleidoscope Day 1

Filed under: Event, Oracle — kocakahin @ 3:07 pm
Tags: , , ,

odtugHi there,

After a long trip to San Jose (around 24 hours :) ), today we have ride to Monterey with my friend Anil. Monterey is a very quite, beautiful city near by the ocean. Although it is in June, Monterey was extremely cold today. We have walked on the sand at sea shore, eat something in a fish restaurant and walk around the city town.

Our hotel Montery Hyatt Regency is one of the best hotels I have ever been. Nice green golf fields, quite environment, and smiling face staff (Thanks Joe for your advices about the down town).

3 hours ago, I have woke up for a phone call (at 2 am in the morning). And my manager was on the phone asking for help on one of the systems I have recently engineered. After solving the problem in the RAC cluster, I had started to write to share my tomorrow (Sunday) program in Kaleidoscope. It seems that I will be attending to APEX stream for whole day. Especially I am curios about following sessions:

  • Keynot Session by Joel Kalman, Oracle Corp
  • Leveraging Oracle 10gXE, APEX, and SQL*Developer to Build  an MDM Solution by Shravan Kumar, APEXor
  • APEX Panel Discussion

See you in Kaleidoscope Day 1

April 12, 2009

Our Data Warehouse Podcast

Filed under: Datawarehousing, Oracle — kocakahin @ 9:15 pm
Tags: , ,

 Hi there,

The podcast we have recently performed is available. Click here…

March 31, 2009

SQLAlchemy for Oracle ORM in Python

Filed under: Oracle — kocakahin @ 2:55 am
Tags: , , , , ,

smilingpythonHi there,

I am trying to learn Python for some daily scripting activities on Oracle databases. I hope you all familiar with ORM tools available for other programming languages. I have already found a very developer friendly ORM tool for Python called SQLAlchemy. In this post you will find a basic usage of this tool to write better Oracle-Python codes. We will be going through the answer of a simple question:

How to compute an ASM Diskgroup Read Rate?

with my special thanks to Anil Chalil…

Blog at WordPress.com.