# Category Archives: Uncategorized

## Oracle Enterprise Linux for Home User: Install Latest Firefox Version

It is a joy to use Linux for any purpose. Although Oracle Linux is mainly designed for enterprise stack, a stable operating system for a technical user is always a need.

However default configurations and program versions are not suitable for daily use always. One example of this is Firefox 3.0.6 available in Oracle Linux (or Red Hat). Here is how you can upgrade it to 17.0

Download latest Firefox release from Mozilla web page. Then unpack and install it by simply following.

tar -xjvf firefox-17.0.tar.bz2 cp -r firefox /opt/ ln -sf /opt/firefox/firefox /usr/bin/firefox

## Line of Sight (LoS) Analysis: Optimizing the Observers for Best Coverage (Part 4)

*N*observers on terrain such that visible region (as many

*green*points as possible by our convention) is maximized ?”.

We first define a pseudo code in order to find the optimal (Not guaranteed. Keep in mind that optimization problems are usually NP-complete by their nature) layout of *N* observers. For simplicity we will assume that all observers have the same height (7 units) which can be relaxed later.

We will implement a constructive way of finding optimal layout for N observers. Here is the pseudo code:

- Find the optimal layout for 1 observer and compute coverage ratio (best coverage for one observer)
- Add another random observer ((
*uniform(-8,8),*) and compute the coverage for those two observers (random one and the best observer from Step 1).*uniform(-8,8))* - If the new coverage is better than the coverage in Step 1, use this as the input of optimization solver
- Otherwise repeat Step 2 to find a better coverage.
- For number of observers greater than 2 apply the idea in Step 2 recursively.

There are some blur points in this pseudo code. We will define those before moving further with the implementation.

### Coverage

The very first thing to be defined is the coverage idea. As you will remember from second post, we have defined our 3D terrain by evaluating our *height* function over outer product of *x & y* values varying over *[-8,8]* with a step size of *0.1* units. We have *1681* different* (x,y)* tuples. Here is the definition of coverage based on our conventions:

**Coverage Ratio**is the ratio of points within LoS of a given observer/group of observers (at least one of the observers mark those set of points as*green*) to the total number of points (1681)

### How to Find Optimal Coordinates of Observers ?

Optimality is a very common word used in place of many different concepts in real life or engineering. Let me define it once more for our purpose:

**Optimization**is the process of searching for an*N-dimensional vector*using a*technique*to maximize/minimize*a function of that N-dimensional vector*.

Now let’s substitute three italic words of definition for our problem:

**N-dimensional vector**in our problem is the vector of first to components of observer dimensions. Such as,*(x1,y1,x2,y2,…,xn,yn).***Technique**to be used is the Nelder and Mead Technique (A version of it implemented in R).**Function**to be maximized is the coverage function which we have defined for a given set of observers.

### Implementation

Let’s start by defining the function to be optimized that is *coverage* of terrain for a given set of observers.

targetfunc<-function(observer){ m <- matrix(data=observer,ncol=2,byrow=TRUE) # Compute merged status of all observers mergedstatus <- rep("red",length(terrain$height)) for(oidx in seq(1:dim(m)[1])){ terrain$dist2observer <- distance(terrain, c(m[oidx,],7)) status <- LoS(terrain,c(m[oidx,],7),maxVisibleDistance) mergedstatus <- updatestatus(mergedstatus,status) } sum(mergedstatus=="green")/1681 }

*matrix* routine allows us to create a table of two columns(first two dimensions of observers) and *length(observer)/2 *rows*. *We have used the technique discussed in part 3 to compute merged status of observers. *sum(mergedstatus==”green”)* call is used to count number of *green* points on terrain with respect to observers.

Next is the computation of first input to be given to optimization solver. That’s because for any optimization technique starting point is critical. Without any formal definition we will use our pseudo code to choose a *“good starting point/vector”.*

n <- 2 baselineValue <- 0.541344 previousObserver <- c(1.15861411217711, 1.1499851362913) observers <- c(previousObserver,runif(2,-8,8)) while(targetfunc(observers) <= baselineValue){ observers <- c(previousObserver,runif(2,-8,8)) } print(observers)

Above code is an example to initialize *observers* vector for searching best 2 observer layout. It uses the best coverage ratio for 1 observer case (*54.1344%*) and adds a new random observer next to best observer found for single observer case.

Final point is the optimization solver which is very simple and totally handled by R

optim <- optim(observers, targetfunc, control=list(fnscale=-1,trace=5,REPORT=1))

First parameter is the initial value for input vector (prepared by previous code piece). Second parameter is the name of the function to be maximized. *optim* function is implemented to solve minimization problems by default. Setting *fnscale* attribute of *control* parameter turns it to a maximization problem solver.

Now we can combine all to have our final script

library(rgl) ################## # Functions ################## # 3D Terrain Function height <- function (point) { sin(point$x)+0.125*point$y*sin(2*point$x)+sin(point$y)+0.125*point$x*sin(2*point$y)+3 } # Linear Function linear <- function (px, observer, target) { v <- observer - target y <- ((px - observer[1])/v[1])*v[2]+observer[2] z <- ((px - observer[1])/v[1])*v[3]+observer[3] data.frame(x=px,y=y, z=z) } # Linear Function distance <- function (terrain, observer) { sqrt((terrain$x-observer[1])^2+(terrain$y-observer[2])^2+(terrain$height-observer[3])^2) } LoS <- function(terrain, observer, maxVisibleDistance){ status = c() for (i in seq(1:nrow(terrain))) { if (observer[1] == terrain$x[i] && observer[2] == terrain$y[i]){ if(observer[3] >= terrain$height[i]){ if (terrain$dist2observer[i] > maxVisibleDistance){ status <- c(status,"yellow") }else{ status <- c(status,"green") } }else{ status <- c(status,"red") } }else{ # All points on line line <- linear(seq(from=min(observer[1],terrain$x[i]), to=max(observer[1],terrain$x[i]), by=0.1), observer, c(terrain$x[i],terrain$y[i],terrain$height[i])) # Terrain Height h <- height(line) # LoS Analysis aboveTerrain <- round((line$z-h),2) >= 0.00 visible <- !is.element(FALSE,aboveTerrain) if (visible){ # Second Rule if(terrain$dist2observer[i] <= maxVisibleDistance){ status <- c(status,"green") }else{ status <- c(status,"yellow") } }else{ status <- c(status,"red") } } } status } updatestatus <- function(status1,status2){ mergedstatus<-c() for(i in seq(length(status1))){ if (status1[i] == "green" || status2[i] == "green"){ mergedstatus <- c(mergedstatus,"green") }else if (status1[i] == "yellow" || status2[i] == "yellow"){ mergedstatus <- c(mergedstatus,"yellow") } else{ mergedstatus <- c(mergedstatus,"red") } } mergedstatus } ################## # Input ################## # Max visible distance maxVisibleDistance = 8 # Generate points with a step size of 0.1 x <- seq(from=-8,to=8,by=0.4) xygrid <- expand.grid(x=x, y=x) terrain <- data.frame(xygrid, height=height(xygrid) ) targetfunc<-function(observer){ #print(observer) m <- matrix(data=observer,ncol=2,byrow=TRUE) # Compute merged status of all observers mergedstatus <- rep("red",length(terrain$height)) for(oidx in seq(1:dim(m)[1])){ terrain$dist2observer <- distance(terrain, c(m[oidx,],7)) status <- LoS(terrain,c(m[oidx,],7),maxVisibleDistance) mergedstatus <- updatestatus(mergedstatus,status) } sum(mergedstatus=="green")/1681 } n <- 3 baselineValue <- 0.541344 previousObserver <- c(-1.32661956044593, 2.18870625357827) # List of observers (x1,y1,z1,x2,y2,z2) observers <- c(previousObserver,runif(2,-8,8)) while(targetfunc(observers) <= baselineValue){ observers <- c(previousObserver,runif(2,-8,8)) } print(observers) optim <- optim(observers, targetfunc, control=list(fnscale=-1,trace=5,REPORT=1))

### Results

Here is the coverage ratio for different number of observers after optimization

You can test covering more than 98% of whole terrain is not trivial by using only 6 random observers but requires “careful” choice of their layout.

Finally let’s check step by step improvement in coverage as we add more optimal observers.

## Single Observer

## Two Observers

## Three Observers

## Four Observers

## Five Observers

## Six Observers

## Line of Sight (LoS) Analysis: Multiple Observers (Part 3)

In this part of my LoS Analysis series, I will try to extend 3D LoS analysis for multiple observers. Assume that you drop multiple observers into a terrain with the aim of covering it perfectly (100% *green*).

We will reuse R codes used in Part 2. However we need to add a simple code piece to be used to merge Line of Sight results of multiple observers. If a point on terrain is visible by any of the observers that means point is visible, if the point is visible but far from all observers that means point is out of LoS due to distance (marked with *yellow*), for all other conditions point on terrain is *red*. *updatestatus* function is implemented for this purpose.

library(rgl) ################## # Functions ################## # 3D Terrain Function height <- function (point) { sin(point$x)+0.125*point$y*sin(2*point$x)+sin(point$y)+0.125*point$x*sin(2*point$y)+3 } # Linear Function linear <- function (px, observer, target) { v <- observer - target y <- ((px - observer[1])/v[1])*v[2]+observer[2] z <- ((px - observer[1])/v[1])*v[3]+observer[3] data.frame(x=px,y=y, z=z) } # Linear Function distance <- function (terrain, observer) { sqrt((terrain$x-observer[1])^2+(terrain$y-observer[2])^2+(terrain$height-observer[3])^2) } LoS <- function(terrain, observer, maxVisibleDistance){ status = c() for (i in seq(1:nrow(terrain))) { if (observer[1] == terrain$x[i] && observer[2] == terrain$y[i]){ if(observer[3] >= terrain$height[i]){ if (terrain$dist2observer[i] > maxVisibleDistance){ status <- c(status,"yellow") }else{ status <- c(status,"green") } }else{ status <- c(status,"red") } }else{ # All points on line line <- linear(seq(from=min(observer[1],terrain$x[i]), to=max(observer[1],terrain$x[i]), by=0.1), observer, c(terrain$x[i],terrain$y[i],terrain$height[i])) # Terrain Height h <- height(line) # LoS Analysis aboveTerrain <- round((line$z-h),2) >= 0.00 visible <- !is.element(FALSE,aboveTerrain) if (visible){ # Second Rule if(terrain$dist2observer[i] <= maxVisibleDistance){ status <- c(status,"green") }else{ status <- c(status,"yellow") } }else{ status <- c(status,"red") } } } status } updatestatus <- function(status1,status2){ mergedstatus<-c() for(i in seq(length(status1))){ if (status1[i] == "green" || status2[i] == "green"){ mergedstatus <- c(mergedstatus,"green") }else if (status1[i] == "yellow" || status2[i] == "yellow"){ mergedstatus <- c(mergedstatus,"yellow") } else{ mergedstatus <- c(mergedstatus,"red") } } mergedstatus } ################## # Input ################## # Observer location #observers<-c(0,0, 6,1,1,6) # Max visible distance maxVisibleDistance = 8 # Generate points with a step size of 0.1 x <- seq(from=-8,to=8,by=0.4) xygrid <- expand.grid(x=x, y=x) terrain <- data.frame(xygrid, height=height(xygrid) ) # List of observers (x1,y1,z1,x2,y2,z2) observers <- c(runif(2,-8,8),6,runif(2,-8,8),6, runif(2,-8,8),6,runif(2,-8,8),6, runif(2,-8,8),6,runif(2,-8,8),6, runif(2,-8,8),6,runif(2,-8,8),6) m <- matrix(data=observers,ncol=3,byrow=TRUE) # Compute merged status of all observers mergedstatus <- rep("red",length(terrain$height)) for(oidx in seq(1:dim(m)[1])){ terrain$dist2observer <- distance(terrain, m[oidx,]) status <- LoS(terrain,m[oidx,],maxVisibleDistance) mergedstatus <- updatestatus(mergedstatus,status) } # Set merged status as the ultimate status terrain <- data.frame(terrain,status = mergedstatus) rgl.open() rgl.surface(x, x, matrix(data=terrain$height,nrow=length(x),ncol=length(x)), col=matrix(data=mergedstatus,nrow=length(x),ncol=length(x)) ) bg3d("gray") # Mark all observers for(oidx in seq(1:dim(m)[1])){ spheres3d(c(m[oidx,1]), c(m[oidx,3]), c(m[oidx,2]), radius=0.25, color="white" ) } rgl.viewpoint(-60,30)

# A Few Examples

Here are a few examples. All those observers are uniformly distributed over terrain using *runif* function

### Trivial Case: Single Observer

### Two Observers

### Four Observers

### Eight Observers

## Line of Sight (LoS) Analysis: 3D Terrain Analysis (Part 2)

In my previous post on LoS Analysis, I have tried to explain briefly the basics of LoS in two dimensional space. Obviously real life problems are based on three dimensional terrains although basic concepts are all the same. In this second part I will try to adapt the same techniques with a few modifications for three dimensional terrains.

### 3D Terrain Visualization with R

One of the first differences in 3D LoS analysis is the terrain visualization. We can not use *plot* function for proper visualization is 3D. Fortunately R has all packages you need for any type of problem. I will use rgl package which can be downloaded using `install.packages("rgl")`

command.

Once you have the *rgl* package, generating pseudo 3D terrains as we did for 2D is a trivial thing.

You can use the following R script to generate your 3D terrains like above.

library(rgl) # 3D Terrain Function height <- function (x,y) { sin(x)+0.125*y*sin(2*x)+sin(y)+0.125*x*sin(2*y)+0.25 } # Terrain boundaries -8<=x<=8 and -8<=y<=8 boundary <- c(-8,8) # Terrain grid with a step size of 0.1 units xy<-seq(from=boundary[1],to=boundary[2],by=0.1) # Evaluate all heights for all grid points z<-outer(xy,xy,height) # A few visualization staff zlim <- range(z) zlen <- zlim[2] - zlim[1] + 1 colorlut <- terrain.colors(zlen) # height color lookup table col <- colorlut[ z-zlim[1]+1 ] # assign colors to heights for each point # Draw the terrain rgl.open() bg3d("gray") rgl.surface(xy, xy, z, color=col)

A new function in this script is *outer* function which generates the product of a vector and a *row-vector* to have a matrix (product of a *row-vector* with a *vector/column-vector* is obviously a scalar value and named to be *dot/inner product*). The third parameter of the function provides us the mechanism to apply a given function (*height* in our case) for each element of this matrix. Obviously you can play with *height *function to have fancier 3D terrains and to have best visualization you may need *viewpoint* routine in rgl package .

### LoS in 3D Terrain

Line of Sight analysis on 3D terrain uses the same principles as it does in 2D. Use the following R script to decide on status of a point (invisible, visible, visible but far away)

library(rgl) ################## # Functions ################## # 3D Terrain Function height <- function (x,y) { sin(x)+0.125*y*sin(2*x)+sin(y)+0.125*x*sin(2*y)+0.25 } # Linear Function linear <- function (x, observer, target) { v <- observer - target y <- ((x - observer[1])/v[1])*v[2]+observer[2] z <- ((x - observer[1])/v[1])*v[3]+observer[3] data.frame(x=x,y=y, z=z) } # Linear Function distance <- function (p0,p1) { sqrt(sum((p0-p1)^2)) } ################## # Input ################## # Observer location observer<-c(10,10,1) # Target on terrain target <- c(5, 5, height(5,5)) # Max visible distance maxVisibleDistance = 4 # Generate points with a step size of 0.1 x <- seq(from=min(observer[1],target[1]), to=max(observer[1],target[1]), by=0.1) # All points on line line <- linear(x, observer, target) # Terrain Height h <- height(line$x,line$y) # LoS Analysis aboveTerrain <- round((line$z-h),2) >= 0.1 # First Rule visible <- !is.element(FALSE,aboveTerrain) if (visible){ # Second Rule d <- distance(observer, target) if(d <= maxVisibleDistance){ status <- "LoS" }else{ status <- "non-LoS due to Distance" } }else{ status <- "non-LoS due to Blocking" }

Obviously there are a few changes in the script with compared to 2D version. The first one is *linear* function(Code Lines 10-18). New version not only evaluates second (*y*) but also the third dimension (*z*). Notice that *z* is our height dimension by convention. We have also utilized *data.frame* function to concatenate all dimensions to form a table of point dimensions

The second difference is on *height* function (Code Lines 5-8). It is no longer a mapping from *x* to *y* but a mapping from* x,y* to *z.*

Rest of the 3D version of script is pretty much the same or trivial to discuss more.

### Visualizing LoS on 3D Terrain

Until this point we have analyzed LoS of a single point on 2D-3D terrains. But usually network analists wish to know LoS map of the terrain with respect to a given observer. In other words we need to visually understand which regions on 3D terrain are visible by the *observer*, invisible by the *observer* due to blocking, or further than the limit from the *observer*.

Here the LoS map of our pseudo 3D terrain with respect to an observer with a given set of coordinates and maximum service range(*green* vs *yellow* regions).

You can obtain this visualization using following R script.

library(rgl) ################## # Functions ################## # 3D Terrain Function height <- function (point) { sin(point$x)+0.125*point$y*sin(2*point$x)+sin(point$y)+0.125*point$x*sin(2*point$y)+3 } # Linear Function linear <- function (px, observer, target) { v <- observer - target y <- ((px - observer[1])/v[1])*v[2]+observer[2] z <- ((px - observer[1])/v[1])*v[3]+observer[3] data.frame(x=px,y=y, z=z) } # Linear Function distance <- function (terrain, observer) { sqrt((terrain$x-observer[1])^2+(terrain$y-observer[2])^2+(terrain$height-observer[3])^2) } LoS <- function(terrain, observer, maxVisibleDistance){ status = c() for (i in seq(1:nrow(terrain))) { if (observer[1] == terrain$x[i] && observer[2] == terrain$y[i]){ if(observer[3] >= terrain$height[i]){ if (terrain$dist2observer[i] > maxVisibleDistance){ status <- c(status,"yellow") }else{ status <- c(status,"green") } }else{ status <- c(status,"red") } }else{ # All points on line line <- linear(seq(from=min(observer[1],terrain$x[i]), to=max(observer[1],terrain$x[i]), by=0.1), observer, c(terrain$x[i],terrain$y[i],terrain$height[i])) # Terrain Height h <- height(line) # LoS Analysis aboveTerrain <- round((line$z-h),2) >= 0.00 visible <- !is.element(FALSE,aboveTerrain) if (visible){ # Second Rule if(terrain$dist2observer[i] <= maxVisibleDistance){ status <- c(status,"green") }else{ status <- c(status,"yellow") } }else{ status <- c(status,"red") } } } status } ################## # Input ################## # Observer location observer<-c(0.835597146302462, -1.71025141328573, 6) # Max visible distance maxVisibleDistance = 8 # Generate points with a step size of 0.1 x <- seq(from=-8,to=8,by=0.4) xygrid <- expand.grid(x=x, y=x) terrain <- data.frame(xygrid, height=height(xygrid) ) terrain <- data.frame(terrain, dist2observer=distance(terrain, observer) ) terrain <- data.frame(terrain, status = LoS(terrain, observer, maxVisibleDistance)) rgl.open() rgl.surface(x, x, matrix(data=terrain$height,nrow=length(x),ncol=length(x)), col=matrix(data=terrain$status,nrow=length(x),ncol=length(x)) ) bg3d("gray") # Mark the observer spheres3d(c(observer[1]), c(observer[3]), c(observer[2]), radius=0.5, color="white" ) rgl.viewpoint(-60,30)

For a better visualization R allows you to implement spinning 3D terrains using *play3d* function and record it in gif format using *movie3d* function as I did below.

## Line of Sight (LoS) Analysis: Basics (Part 1)

# Introduction

Line of Sight analysis is a commonly used technique in telecommunication industry for A/I (Air Interface) equipment planning and allocation. With the simplest terms LoS is the question whether a point on N-dimensional space is visible by an other observer point. The question can be used to answer where to locate a transceiver on terrain so that it can serve customers on some region A.

Before relatively more complicated problems, let’s start with an easy example focusing on two dimensional terrains. Throughout the post, we will use R for coding which is my favorite option for any mathematical problem (statistics, plotting, linear algebra, optimization, etc.). But you can easily adapt coding material to Mathlab, Python,or your favorite language.

We will start by defining a mathematical function to be used to generate our pseudo terrains. For this purpose trigonometric functions (*sin*, *cos*) and polynomial functions are the best ones because of their wavy shapes. Here is an example of trigonometric terrain

**Figure 1 Trigonometric Terrain
**

In order to generate this two dimensional one use the following code piece

x <- seq(from=4,to=10,by=0.01) y <- sin(x)+cos(2*x)+sin(3*x)+cos(4*x)+3 windows() plot(x,y,'l', main="y=sin(x)+cos(2x)+sin(3x)+cos(4x)+3", ylab="height",col="blue")

**Figure 2 Polynomial Terrain
**

To obtain this terrain, use the following R script piece

x <- seq(from=0,to=6,by=0.01) y <- x*(x-1)*(x-2)*(x-3)*(x-4)*(x-5)*(x-6)+100 windows() plot(x,y,'l', main="y=x(x-1)(x-2)(x-3)(x-4)(x-5)(x-6)+100", ylab="height",col="blue")

Combining polynomial terrain functions with trigonometric ones will give you fancier ones.

# What is LoS ?

You can think that we have already answered this question but this was an informal try which is not very useful for solving the problem. In order to solve this problem methodically we need to understand what makes a *target* visible (within LoS) by the *observer*.

As you see on *Figure 3*, **green** point is within line of sight of *observer* (**blue** point). However there is pseudo hill between **red** point and *observer*. The difference is that the line connecting *observer* and **green** point is always greater than the terrain function whereas this is not valid for the line connecting *observer* and **red** point (for x ε [~2.5, ~3.5] red line is under the terrain curve).

**Figure 3 LoS vs non-LoS
**

This was the first point (blocking) we should define. The second one is an easier one related with maximum Euclidean distance between *observer* and *target*. The distance between *observer* and *target* may cause a phase shift in signal if the distance is sufficiently long or depending on weather conditions and terrain properties you may observer diffraction problems (actually there might be more than those). In return this will cause signal quality issues or call drops. On Figure 3, although *blocking* is not an issue between *observer* and **yellow** point, *target* is out of visible range (say 8 units) of *observer*.

You can generate *Figure 3* using the following R script

# Terrain Function height <- function (x) { x*x/3+sin(x)+cos(2*x)+sin(3*x)+cos(4*x)+sin(5*x)+cos(6*x)+3 } # Observer location observer<-c(1.5,8.9) # Generate terrain points with a tolerance of 0.1 x<-seq(from=-0.1,to=6.1,by=0.1) terrainHeight<-height(x) windows() # Draw terrain plot(x,terrainHeight,type='b', xlim=range(x),ylim=range(terrainHeight), main="Line of Sight (LoS)", ylab="Height",xlab="") # Not LoS points(x=c(observer[1],x[41]), y=c(observer[2],terrainHeight[41]), col="red",type='b') # LoS points(x=c(observer[1],x[5]), y=c(observer[2],terrainHeight[5]), col="green",type='b') # LoS but far points(x=c(observer[1],x[length(x)]), y=c(observer[2],terrainHeight[length(x)]), col="yellow",type='b') # Draw Observer points(x=c(observer[1]), y=c(observer[2]), col="blue",pch=10)

# Method to Decide LoS

Finally let’s define a method to find all visible, invisible, and “far” points on any terrain. Since it is not “easy” to decide analytically whether the line connecting *observer* and *target* “is above” the terrain for any terrain function, we will use a simple numeric method.

We will define a *step* size small enough (around *Spatial Tolerance*) to generate all *x* values between *observer* and *target*. *seq* function is a good choice for doing this (Code Lines 33-36). Evaluate these *x* values for line function connecting *observer* and *target *and terrain function. Evaluation is simple for terrain function using *height* function (Code Lines 4-7). Evaluation of line function is held by function *linear* using parametric definition of line function (Code Lines 9-14) . Next step is to search for any *x* value having a line evaluation less than terrain evaluation (Code Line 44-28). The rest is simple as to evaluate euclidean distance and assigning values to *status* variable.

################## # Functions ################## # Terrain Function height <- function (x) { x*x/3+sin(x)+cos(2*x)+sin(3*x)+cos(4*x)+sin(5*x)+cos(6*x)+3 } # Linear Function linear <- function (x, observer, target) { v <- observer - target ((x - observer[1])/v[1])*v[2]+observer[2] } # Linear Function distance <- function (p0,p1) { sqrt(sum((p0-p1)^2)) } ################## # Input ################## # Observer location observer<-c(1.5,9) # Target on terrain target <- c(5, height(5)) # Max visible distance maxVisibleDistance = 4 # Generate points with a step size of 0.1 x <- seq(from=min(observer[1],target[1]), to=max(observer[1],target[1]), by=0.1) # Terrain Height h <- height(x) # y Values y <- linear(x, observer, target) # LoS Analysis aboveTerrain <- round((y-h),2) >= 0.00 # First Rule visible <- !is.element(FALSE,aboveTerrain) if (visible){ # Second Rule d <- distance(observer, target) if(d <= maxVisibleDistance){ status <- "LoS" }else{ status <- "non-LoS due to Distance" } }else{ status <- "non-LoS due to Blocking" }

## My Comments on Voltaire InfiniBand (IB)

As you all may know Voltaire is one of the most prominent IB technology vendors and investors had the partnership with Oracle in building Exadata Version 1. Yet another beautiful thing about Voltaire is that it is one of the companies invest on Open Source technologies and HPC (High Performance Computing) a lot, like OFED stack development on Linux.

You can find my comments about Voltaire 20Gbit IB solution which we have already used in our large (25TB) RAC consolidation project on Oracle 11g Release 1.

Business Worldwide Choose Voltaire InfiniBand to Enhance Performance and Scalability of Oracle RAC

## ORION IO Calibration Cookbook

ORION is one of the most important calibration tools of my toolbox. In any project before installing any Oracle software (clusterware, ASM or RDBMS), the first thing I do is to calibrate disks of the system. That’s because history is full storage related Oracle stories.

In this post, you will find a simple cookbook of calibrating a storage array by using ORION. I will go through a real calibration test I did three months ago.

# Preparing a .lun file

In any ORION test, the first thing you should do is to prepare a LUN file with extension *.lun *to list your devices (with the assumption that you will be testing raw devices. ORION also let you to test file systems in which you should use synthetically created large files in place of device names) to be calibrated. Here is our *lun* file

[oracle@consol10g orion]$ cat mytest.lun/dev/dm-2 /dev/dm-3 /dev/dm-4/dev/dm-5 /dev/dm-6 |

One important point is that you should ensure that the OS user you will be performing the test has the necessary grants over the devices you will be calibrating.

# Small Random & Large Sequential Read Load

I always start with pure small (8K) random and then pure large (1M) sequential read load calibration first. That is simply because if these two tests fail to satisfy your performance needs there is no point in doing any other tests. You can simply call back your vendor or infrastructure team.

Here is the ORION syntax we will be using

[oracle@consol10g orion]$ ./orion_lnx -run advanced -testname mytest -num_disks 40 -simulate raid0 –write 0 –type seq –matrix basic –cache_size 67108864 -verbose |

Let us discuss this parameterization. In order to control other options in detail I have set **run** option to be *advanced*. **testname** option should set to be the name of *.lun* file. **num_disks** is the number of disks in your storage array. Actually, you can set this parameter to any value you like it is just an input for calculating the maximum number of small IO and large IO requesters (for value less than 10 max small IO requesters= 5***num_disks** and max large IO requesters = 2***num_disks**).**simulate** parameter is always *raid0* me, because I always use ASM. **type** option defines the type of large IOs (sequential/random). Since the test covers pure random and pure sequential IO loads, **matrix** option is set to be *basic*. **cache_size** is a critical parameter you should learn by get in touch with your storage admin. This is the size of your storage array cache in MB. If you set this parameter too low your outputs will be too good to be true, if you set too high your test run time will be too long.** verbose** is the parameter to guide ORION to print progress status on standard output.

Keep in mind that you should run ORION from the same directory where *mytest.lun* file resides. When the execution is over you will find three new files in your current working directory. Those are *mytest_*_summary.txt, mytest_*_iops.txt, mytest_*_lat.txt, mytest_*_mbps.txt* files.

**Figure 1 Pure Random and Pure Sequential Loads**

As you see the number of read IOPS is increasing with the increasing number of outstanding random read requesters to a certain point. This is what we expect to see from queuing theory. However, there is another restrictive parameter over that, called *latency*. If the service time of a single read request exceeds 10ms on the average, (some others say 20ms) then the user will start to suffer.

So keep in mind that doing ~100.000 IOPS means nothing. What is meaningful is that you can deliver ~3500 IOPS with 19.5 ms service time. In other words never use IOPS metric barely. Always cascade average service time, latency, queuing time, etc metrics.

The interpretation of sequential IO is different. In high throughput demanding systems, nobody will deal with the service time of a single IO request. The import thing is to fully utilize the whole storage infrastructure in order to deliver the highest rate of throughput possible. Therefore, “160 MB/s read throughput” statement is OK.

# Mix Read Load

The last thing you should test is the behavior of storage array under various mix loads. Let me exemplify the importance of this test. You might see some systems running in peace whole day long but 1-2 hours night time (backup window). Alternatively, you might see some DBAs periodically kill reporting users in OLTP systems.

Those are all related with insufficient IO configurations. Large IO requests caused by backup, reporting,etc activites may result in a severe change in the service time of small IO requests. Therefore, you should have a priori knowledge in what proportion, small and large IO requests can cause a problem on your storage infrastructure. Here is the ORION syntax

[oracle@consol10g orion]$ ./orion_lnx -run advanced -testname mytest -num_disks 40 -simulate raid0 –write 0 –type seq –matrix detailed –cache_size 67108864 -verbose |

Notice that the only change in here is that **matrix** option is set to be *detailed. *When **matrix** option is *basic,* ORION generates random IO at N different levels without sequential IO requesters and then it generates sequential IO at M different levels without random IO requesters. When the option equals to *detailed*, ORION generates all possible MxN combination of random & sequential IO genererators.

**Figure 4 Mix Read Load**

To interpret Figure 4, let’s think that our storage array is capable of serving only 8K requests. Any larger requests will be chopped into 8K pieces. That means a large IO request will be corresponding to 125 small IO requests. Moreover think that the total capacity of our storage array is 2000 small IOPS. Now by simple division you can either yield 2000 small (8K) IOPS or 16 large (1M) IOPS from this storage array or somewhere between.

So when the number of total large IO requesters increase, the number of total IOPS will decrease.

Now assume that sustaining 1500 IOPS requires 10 ms, 3000 IOPS requires 20 ms service time on the average. While we are sustaining 1500 IOPS, we can either move on large requester axis and with an addition of 12 IOPS we can reach 20 ms latency, or we can move on small requester axis and with an addition of 1500 IOPS we can reach 20 ms latency (We may choose a third option somewhere between also). As a result increase in large IO results in an increase in service time also.

# Conclusion

In this post, I try to show how ORION can help you in detecting your possible disk performance problems before they occur on production. Remember that on an enterprise storage infrastructure there are more than just a bunch disks. The major performance problem can be related to components like HBAs, switches, port issues, backend problems, or even IO scheduling algorithm you use.

# Mathlab Codes to Create Charts

In order to generate the plots you’ve seen in this post you can use *orion_chart_basic.m* and *orion_chart_detailed.m* scripts with mathlab. When you execute them(F5), they will ask you to choose a directory. This directory should contain all ORION output files of a single run. Once you pick the directory, mathlab will do the rest.

## CAUTIONS

- For tests executed with
**run**option set to be*basic*you should use*orion_chart_basic.m* - For tests executed with
**run**parameter set to be*detailed*you should use*orion_chart_detailed.m* - Each directory you choose should contain one and only one set of ORION run output file.
- All four mathlab files should be in the same working directory.

## randomIOChart.m

function randomIOchart(concSmallIO, IOPS, latency) [haxes,hline1,hline2] = plotyy(concSmallIO,IOPS,concSmallIO,latency,’plot’); axes(haxes(1)); xlabel(‘Outstanding Small IO’) ylabel(‘IOPS’); title(‘Random I/O’,’FontSize’,14) j = 1; [m n]=size(latency) while latency(j) < 20 j = j + 1; if n == j, break; end end
text(concSmallIO(j-1),IOPS(j-1),[int2str(IOPS(j-1)),’ IOPS with ‘,num2str(latency(j-1)),’ ms latency \rightarrow’ ],’HorizontalAlignment’,’right’,’FontSize’,12) axes(haxes(2)); ylabel(‘Latency(ms)’); |

## sequentialIOChart.m

function sequentialIOchart(largeIO,mbps) plot(largeIO,mbps,’-rs’,’LineWidth’,2,… ‘MarkerEdgeColor’,’k’,… ‘MarkerFaceColor’,’g’,… ‘MarkerSize’,10) xlabel(‘Outstanding Large IO’) ylabel(‘MB/s’) title(‘Sequential I/O’,’FontSize’,14) |

## orion_chart_basic.m

%Use this file for ORION test output executed with -run basic option %When the directory selection pop appears, pick the folder that contains %ORION run outputs. orion_directory = uigetdir(); latency_file = dir(fullfile(orion_directory,’*_lat.csv’)); iops_file = dir([orion_directory,’\*_iops.csv’]); mbps_file = dir([orion_directory,’\*_mbps.csv’]); latency = csvread(fullfile(orion_directory,latency_file.name), 1, 1) latency = latency(1,:) small = csvread(fullfile(orion_directory,latency_file.name), 0, 1); small = small(1,:)
iops = csvread(fullfile(orion_directory,iops_file.name), 1, 1); iops = iops(1,:);
subplot(2,1,1); randomIOchart(small, iops, latency)
mbps = csvread(fullfile(orion_directory,mbps_file.name), 1, 1); large = csvread(fullfile(orion_directory,mbps_file.name), 1, 0); large = large(:,1); subplot(2,1,2); sequentialIOchart(large,mbps); |

## orion_chart_detailed.m

%Use this file for ORION test output executed with -run detailed option %When the directory selection pop appears, pick the folder that contains %ORION run outputs. orion_directory = uigetdir(); latency_file = dir(fullfile(orion_directory,’*_lat.csv’)); iops_file = dir([orion_directory,’\*_iops.csv’]); mbps_file = dir([orion_directory,’\*_mbps.csv’]); latency = csvread(fullfile(orion_directory,latency_file.name), 1, 1); small = csvread(fullfile(orion_directory,latency_file.name), 0, 1); small = small(1,:); large = csvread(fullfile(orion_directory,latency_file.name), 1, 0); large = large(:,1);
subplot(2,2,1);surf(small,large,latency);xlabel(‘Outstanding Small IO’);ylabel(‘Outstanding Large IO’);zlabel(‘Latency (ms)’);
iops = csvread(fullfile(orion_directory,iops_file.name), 1, 1); subplot(2,2,2);surf(small,large,iops);xlabel(‘Outstanding Small IO’);ylabel(‘Outstanding Large IO’);zlabel(‘IOPS’);
mbps = csvread(fullfile(orion_directory,mbps_file.name), 1, 1) small = csvread(fullfile(orion_directory,mbps_file.name), 0, 1); small = small(1,:) large = csvread(fullfile(orion_directory,mbps_file.name), 1, 0); large = large(:,1) subplot(2,2,[3 4]);surf(small,large,mbps);xlabel(‘Outstanding Small IO’);ylabel(‘Outstanding Large IO’);zlabel(‘MB/s’); |

## Welcome to Microsoft Office 2007

Since my new company *Krom Technology Services A.Ş.* is a Microsoft partner, I had the chance to experience new Office Enterprise 2007. The only thing can be said about new Office is that Microsoft will never bankrupt unless it gives up Office.

New Office not only brings new applications like *Microsoft Groove 2007* (http://office.microsoft.com/assistance/asstvid.aspx?assetid=XT100627131033&vwidth=700&vheight=530&type=flash&CTT=11&Origin=HA101672641033 ), but also Microsoft completely changes how the Office looks. It has to be related with new *Vista* environment most probably. Old menu bar (File, View, etc.) has been removed and new marshmallow style user menus are presented instead. You can find whatever you search immediately. Moreover you can customize them as you like.

One thing that really beats me is new equation editor Microsoft Office 2007 provides. You don’t need to use Equation Editor 3.0 explicitly any more. You just press ** Alt + =** while you are writing your text and equation editor is opened for you and you embed whatever equation you like into your work.

Other good news is for Adobe Acrobat Professional users. You don’t need Adobe Professional any more in order to convert Office documents into Portable Document Files (pdf). That’s because Office 2007 brings built-in support for pdf and XML Paper Specification (XPS).

Another good thing is Microsoft really improves it crash recovery structure for Office applications. You remember nasty crashes while you are writing you projects. Right? New Office 2007 has a more strong resistance against document corruption.

Besides virtual changes file extensions are also changed in Office 2007. Good old Word documents have *docx* extensions, Excel files have xlsx extensions. As you may understand all old file extensions are padded with an *x* letter. Older file Office files can easily be opened but in order to use new features like this new equation editor you need to convert you files to this new format.

One good new is for bloggers, I have prepared this blog post in Office 2007 Word and just click on some buttons to publish it. There exist build-in supports for various blog accounts like wordpress, BlogSpot, etc.

Something came weird to me is that new Office 2007 is not any slower than Office 2003 J

Those are my a few experiences on Microsoft Office 2007 (especially Word, Excel). I have written my senior project report using Microsoft Office 2007 (I can hear the ones saying “This was a risk”J) and convert it to pdf and I will be submitting it tomorrow. It was a fast and good job with the help of Microsoft Office 2007.

You know me. I’ve never been a Microsoft guy, except that I like working with Office. And this new Office is really good.

## My master has just moved…

My Yoda has just moved to WordPress in this week. The only thing left to me is to perform the ritual. Following his foot steps…