MacOSX PostgreSQL post-install setup

I've decided that my new Macbook Pro should henceforth be my primary development machine. This meant that I had to install PostgreSQL as one of my long list of dev tools. I was very surprised to find that my Fink -installed PostgreSQL was pretty much non-functional after the install. This article will walk you through the post-install settings on MacOSX (I used 10.5, but presumably this should work for all other versions) to get a proper running PostgreSQL.


The first thing we must do is fix up the "postgres" user's system settings. After installation you won't be able to "su" to the "postgres" user because that user's shell conveniently points at "/dev/null". MacOSX's user account management GUI only allows editing of desktop "login" accounts (those with uid > 500 ). We will therefore have to recruit MacOSX's dscl commandline tool for this job. You can learn more about dscl here.

Open a terminal window and type:
$sudo dscl .
> read /Users/postgres
AppleMetaNodeLocation: /Local/Default
AuthenticationHint:
NFSHomeDirectory: /var/empty
Password: *
PrimaryGroupID: 252
RealName:
PostgreSQL Database Server
RecordName: postgres
RecordType: dsRecTypeStandard:Users
UniqueID: 252
UserShell: /dev/null


You will notice that several of the account settings are awry. Specifically we must fix "UserShell" and "NFSHomeDirectory".

Before we can fix the NFSHomeDirectory of the user, let's create a convenient symbolic link :


$sudo ln -s /sw/var/postgres-xx  /sw/var/pgsql

Now once more we run the "dscl" tool, and use the "create" command to update the "postgres" user's home directory:
$sudo dscl .
> create /Users/postgres NFSHomeDirectory /sw/var/pgsql
> read /Users/postgres
AppleMetaNodeLocation: /Local/Default
AuthenticationHint:
NFSHomeDirectory: /sw/var/pgsql
Password: *
PrimaryGroupID: 252
RealName:
PostgreSQL Database Server
RecordName: postgres
RecordType: dsRecTypeStandard:Users
UniqueID: 252
UserShell: /dev/null
>

We do the same for the postgres user's shell :
> create /Users/postgres UserShell /bin/bash
> read /Users/postgres
AppleMetaNodeLocation: /Local/Default
AuthenticationHint:
NFSHomeDirectory: /sw/var/pgsql
Password: *
PrimaryGroupID: 252
RealName:
PostgreSQL Database Server
RecordName: postgres
RecordType: dsRecTypeStandard:Users
UniqueID: 252
UserShell: /bin/bash
>

We now "su" to the postgres user and initialize our database. IMPORTANT: Don't do this step if you have existing data in "/sw/var/pgsql/data" or you will LOSE ALL YOUR DATA!
$/sw/bin/initdb -D /sw/var/pgsql/data -E utf8

Once this is done you can start your PostgreSQL instance with

$/sw/bin/pg_ctl -D /sw/var/pgsql/data start


You will have to manually stop/start your PostgreSQL instance every time. If you want to start your service automatically at boot or login time, you might want to create a StartupItem. You can learn more about MacOSX StartupItems here:



Read more...

Android Hello NDK

This is a very quick introduction to building JNI (Java Native Interface) projects using the recently released Android NDK (Native Development Kit). My build system runs Linux, so if you're on Windows or Mac YMMV. You can download the full sources, including the NDK files as well as an Eclipse project at the bottom of this article.

The Java Native Interface allows developers to access the methods inside non-Java shared libraries (.dlls on MS Windows and .so files on *nix) from within your Java applications. The Android NDK is essentially JNI for the Dalvik VM.


If you have not already done so, please download and install the NDK from here: Android NDK Download


Creating our Activity

Let's start by creating an activity which will load our shared library object via a call to the "System.loadLibrary()" method inside a static initializer block. We then invoke our native method inside our activity's "onCreate()" method. At this point we won't actually have a shared library to load, we'll get to that later.

"<project_home>/src/net/bruary/hellondk/HelloNDK.java

 8 public class HelloNDK extends Activity {
9 TextView txtHello;
10
11 /** Called when the activity is first created. */
12 @Override
13 public void onCreate(Bundle savedInstanceState) {
14 super.onCreate(savedInstanceState);
15
16 setContentView(R.layout.main);
17
18 //obtain a reference to our textview control
19 txtHello = (TextView) findViewById(R.id.txtHello);
20
21 //invoke our native method
22 String hello = sayHello();
23
24 //update our textview control
25 txtHello.setText(hello);
26
27 }
28
29 //Our native method. This is needed by the "javah" tool
30 //for generating our .h include header.
31 public native String sayHello();
32
33 //Our static initializer which will load our shared library
34 //into our classloader/VM
35 static {
36 //your library will typically be called "libhellondk.so"
37 //on Linux, but by convention the "lib" prefix, and ".so"
38 //suffix are omitted
39 System.loadLibrary("hellondk");
40 }
41 }



Generating C/C++ header files

In order to generate the C/C++ header file for our shared library we need to:
  1. Build/Compile our Android project so the Java .class files are generated under <project_home>/bin
  2. Use the javah tool from the Java SDK to generate a C/C++ header file for inclusion in our shared object sources
Once you have built your Android project sources, issue the following command at the top of your <project_home>:


$ javah -o hellondk.h -classpath bin net.bruary.hellondk.HelloNDK


The above command will have generated a file called hellondk.h .

Creating our Shared Library

I will assume that the environment variable $NDK_HOME points at the folder where you extracted/installed your Android NDK. Let us now create two folders:

$mkdir
-p $NDK_HOME/sources/hellondk
$mkdir -p $NDK_HOME/apps/hellondk


The "$NDK_HOME/sources/hellondk" folder will contain the source code for your shared/native library.
The "$NDK_HOME/apps/hellondk" folder is the "project definition" folder for your shared library.

Now copy the previously generated header file 'hellondk.h' from your Eclipse project folder to "$NDK_HOME/sources/hellondk/"

In order to eventually build our NDK project, we will need to create a couple of Makefiles. The two Makefiles are:

"$NDK_HOME/sources/hellondk/Android.mk"
LOCAL_PATH := $(call my-dir)
  include $(CLEAR_VARS)

#I Like to build my shared libs using C++
LOCAL_DEFAULT_CPP_EXTENSION := cpp

LOCAL_MODULE :
= hellondk
LOCAL_SRC_FILES := hellondk.cpp

include $(BUILD_SHARED_LIBRARY)

... and ...

"$NDK_HOME/apps/hellondk/Application.mk"

APP_PROJECT_PATH := ~/workspace.android/HelloNDK
APP_MODULES := hellondk


Creating Shared Library Source

Once our makefiles are in place we are left only with creating our C++ source file for the shared library. In the following listing we invoke the JNI NewStringUTF() method to convert a C-string to a JNI jstring object, and return that. See links at the bottom of the article for more on JNI.

$NDK_HOME/sources/hellondk/hellondk.cpp

 #include <hellondk.h>

JNIEXPORT jstring JNICALL Java_net_bruary_hellondk_HelloNDK_sayHello(JNIEnv *env, jobject obj){
return env->NewStringUTF("Hello, NDK!");
}



Compiling your Shared Library

Change directory into the toplevel folder of your NDK installation ($NDK_HOME). Now run the command "make APP=hellondk"
(output of command "
make APP=hellondk" on my machine)

stephan@roadrunner:~/bin/android-ndk-1.5_r1$ make APP=hellondk
Android NDK: Building for application 'hellondk'
Compile++ thumb: hellondk <= sources/hellondk/hellondk.cpp SharedLibrary : libhellondk.so Install : libhellondk.so => ~/workspace.android/HelloNDK/libs/armeabi
stephan@roadrunner:~/bin/android-ndk-1.5_r1$


You can now run your HelloNDK project from within Eclipse. Here's a screenshot of what you should see in the emulator.



You can download the Eclipse Project here: Download Eclipse Project
You can download the NDK project files here: Download NDK Project Files
(Please NOTE: Extract the NDK project files inside you $NDK_HOME folder to have the files deployed into the appropriate places)

Links & Resources:
Android NDK Release 1
JNI Training



Read more...

Android Bluetooth Hacking using the NDK

Many people lamented the omission of the Bluetooth APIs from Android 1.1 and 1.5 firmware builds. This post will detail a method of writing bluetooth-enabled apps for Android using nothing more than the recently released Native Development Kit (NDK). You will not have to root your phone. However, since the NDK is available for firmwares 1.5 onwards only, this method will unfortunately NOT work on 1.1 firmwares. Full source for an Eclipse demo project can be found at the bottom of this article. This article builds on my earlier post (Android Hello NDK).


Obtaining the C Headers and Bluetooth System Libraries

The bad news is that you will have to recompile the 1.5 (CupCake) sources with bluetooth support enabled. The somewhat good news is that you can find ready-built binaries and header files for download at the end of this article. If you prefer to build your own library though:
  1. Follow the build instructions which can be found in the Android Bluetooth FAQ.
  2. Copy the libraries libbluetooth.so and libbluedroid.so into $NDK_HOME/build/platforms/android-1.5/arch-arm/usr/lib/
  3. Copy the bluetooth folder from $SDK_SOURCE/external/bluez/libs/include/bluetooth/ to $NDK_HOME/build/platforms/android-1.5/arch-arm/usr/include/
You are now ready to start building bluetooth-enabled apps. Please note that we only need to place the ARM bluetooth binaries in the NDK to resolve link-time dependencies. We DO NOT actually have to deploy/add the bluetooth libraries to our phones using our .apk file, since the very same bluetooth libraries you built from the are already available on the Android device.

Android and Bluez


Bluetooth support on the Android is provided by the excellent Bluez library. There is however a dearth of good information/documentation on Bluez programming. What documentation there is on Bluez programming seems to mostly cover the Python bindings. There is only one resource that I could find which does a decent job of covering C programming using Bluez (See Resources).

The Android Project

This demo application will do a simple scan for bluetooth devices, and populate an Android ListView control with the results. Let us now create an Activity that includes a ListView, declares a native JNI method, and loads a shared library inside a static initializer.

"<project_home>/src/net/bruary/bluescan/BlueScan.java"


10 public class BlueScan extends Activity {
11
12 ListView lvDevices;
13 private ArrayList deviceArray;
14 private ArrayAdapter listAdapter;
15
16 /** Called when the activity is first created. */
17 @Override
18 public void onCreate(Bundle savedInstanceState) {
19 super.onCreate(savedInstanceState);
20
21 setContentView(R.layout.main);
22
23 lvDevices = (ListView) findViewById(R.id.lvDevices);
24 deviceArray = new ArrayList();
25 listAdapter = new ArrayAdapter(this, android.R.layout.simple_list_item_1, deviceArray);
26 lvDevices.setAdapter(listAdapter);
27
28 String[] devices = btDeviceScan();
29
30 for (String device: devices){
31 deviceArray.add(device);
32 }
33
34 listAdapter.notifyDataSetChanged();
35
36 }
37
38 /*Our native method which receives an array of Strings back from
39 * the C++ shared library
40 * */

41 public native String[] btDeviceScan();
42
43 static {
44 System.loadLibrary("bluescan");
45 }
46
47 }




Setting up our NDK Project

Our NDK project is setup as per usual. The notable difference is in our library's Android.mk file, where we now must include the libbluetooth.so library. In the following listing please pay special attention to line #5. Due to a bug in the current NDK, we must explicitly set our library path to $(SYSROOT)/usr/lib. If you omit this, the linker won't find libbluetooth(or libc, libm etc. for that matter)


 1 LOCAL_PATH := $(call my-dir)
2
3 include $(CLEAR_VARS)
4
5 LOCAL_LDLIBS :
= -L$(SYSROOT)/usr/lib -lbluetooth
6 LOCAL_DEFAULT_CPP_EXTENSION := cpp
7
8 LOCAL_MODULE :
= bluescan
9 LOCAL_SRC_FILES := bluescan.cpp
10
11 include $(BUILD_SHARED_LIBRARY)



Shared Library Sources

The following listing shows our C++ implementation of the Bluez code for scanning for some nearby bluetooth devices. I don't want this to devolve into a Bluez programming tutorial, so I will keep my comments brief.
This sourcecode is adapted from the "simplescan" application in the "Bluetooth Programming in C with Bluez" tutorial. Therefore some of the code is a bit redundant (e.g. perror() is redundant in our NDK app). I also don't do any error/exception handling, so please don't deploy this source to production. You've been warned :)

The two bluez API methods we call are hci_get_route() on line 30, which gives us the deviceID of our Phone's bluetooth device. The other method is hci_inquiry() on line 40, which does the actual scanning. Notice that we pass an array of inquiry_info structs to hci_inquiry().
Please do read the article linked to in the Resources section for more details on Bluez programming.



1 #include <string.h>
2 #include <jni.h>
3 #include <stdio.h>
4 #include <stdlib.h>
5 #include <unistd.h>
6 #include <sys/socket.h>
7 #include <arpa/inet.h>
8 #include <bluetooth/bluetooth.h>
9 #include <bluetooth/rfcomm.h>
10 #include <bluetooth/hci.h>
11 #include <bluetooth/hci_lib.h>
12 #include <bluescan.h>
13
14
15 JNIEXPORT jobjectArray JNICALL Java_net_bruary_bluescan_BlueScan_btDeviceScan(JNIEnv *env, jobject obj)
16 {
17 int max_rsp = 255;
18 inquiry_info* infoArray = new inquiry_info[max_rsp];
19 inquiry_info **ii = NULL;
20 int num_rsp;
21 int dev_id, len, flags;
22 int i;
23 char addr[19] = {0};
24 char name[248] = {0};
25 bool bHasError =false;
26 jstring errMsg;
27
28 jobjectArray btDevices;
29
30 dev_id = hci_get_route(NULL);
31 if (dev_id < 0 ){
32 bHasError = true;
33 errMsg = env->NewStringUTF("Error obtaining device ID");
34 }
35
36 len = 15;
37 flags = IREQ_CACHE_FLUSH;
38 infoArray = new inquiry_info[max_rsp];
39
40 num_rsp = hci_inquiry(dev_id, len, max_rsp, NULL, &infoArray, flags);
41 if (num_rsp < 0)
42 perror("hci_inquiry");
43
44 btDevices = env->NewObjectArray(num_rsp, env->FindClass("java/lang/String"), env->NewStringUTF(""));
45
46 for (i = 0; i < num_rsp; i++){
47 ba2str(&(infoArray[i].bdaddr), addr);
48 memset(name, 0, sizeof(name));
49
50 env->SetObjectArrayElement(btDevices, i, env->NewStringUTF(addr));
51
52 }
53
54 delete(infoArray);
55
56 return btDevices;
57 }




Running the Application

When running the application, please note that I set the len parameter above to 15 (fifteen). The total scanning time will therefore be (1.28 * 15)seconds. I don't know where the 1.28 comes from, much of Bluez is still a mystery to me.

Resources

Blueooth Programming in C with Bluez
You can download the Eclipse project files here: Eclipse Project Files
You can download the NDK project files here: NDK Project Files
You can download my pre-built libbluetooth binaries for ARM NDK here: Bluetooth NDK Binaries


Read more...

Android SMS Handling

I found several nice links [here and here] showing how to do SMS handling in Android. However those examples all seem to be specific to a pre-1.0 version of the SDK. SDK versions 1.1 and 1.5 have a slightly different way of handling things. This blog post will show you how to get an asynchronous notification when new SMS messages arrive on your phone, for the Android 1.1 and 1.5 SDKs. Thanks to Cristina for posting this method to the android-developers group.




If you remember your Android Fundamentals, you will recall that " ... broadcast receivers are activated by asynchronous messages called intents".

The first thing we must therefore do is to register a new broadcast receiver in our "AndroidManifest.xml" file. Additionally we must request the "android.permission.RECEIVE_SMS" permission from the framework. See the following complete listing for "AndroidManifest.xml".

 1 <?xml version="1.0" encoding="utf-8"?>
2 <manifest xmlns:android="http://schemas.android.com/apk/res/android"
3
package="sg.aarvark.android"
4
android:versionCode="1"
5
android:versionName="1.0">

6 <application android:icon="@drawable/icon"
7
android:label="@string/app_name" android:debuggable="true">

8 <activity android:name=".SMSDroid"
9
android:label="@string/app_name">

10 <intent-filter>
11 <action android:name="android.intent.action.MAIN" />
12 <category android:name="android.intent.category.LAUNCHER" />
13 </intent-filter>
14 </activity>
15 <receiver android:name=".SMSReceiver">
16 <intent-filter>
17 <action android:name="android.provider.Telephony.SMS_RECEIVED"/>
18 </intent-filter>
19 </receiver>
20 </application>
21 <uses-sdk android:minSdkVersion="3" />
22 <uses-permission android:name="android.permission.RECEIVE_SMS"/>
23 </manifest>

Now we must create a class that extends BroadcastReceiver. BroadcastReceiver is abstract, and we must override the "onReceive" method.
SMS Messages are delivered to the phone in the PDU (Protocol Description Unit) format. Our goal is therefore to
  1. Get the PDUs representing our incoming SMS message
  2. Decode the PDUs to get readable SMS messages
The following listing for a class called SMSReceiver is all we need.

1 public class SMSReceiver extends BroadcastReceiver {
2
3 private static final String LOG_TAG = "SMS_FUNKY";
4 private static final String ACTION = "android.provider.Telephony.SMS_RECEIVED";
5
6 @Override
7 public void onReceive(Context context, Intent intent) {
8
9 if (intent.getAction().equals(ACTION)){
10 Bundle bundle = intent.getExtras();
11
12 Object[] pdusObj = (Object[]) bundle.get("pdus");
13
14
15 SmsMessage[] messages = new SmsMessage[pdusObj.length];
16 for (int i=0; i <pdusObj.length; i++){
17 messages[i] = SmsMessage.createFromPdu((byte[]) pdusObj[i]);
18
19 }
20
21 StringBuilder sb = new StringBuilder();
22 for (SmsMessage currentMessage: messages){
23 sb.append("Received SMS Message\nFrom: ");
24 sb.append(currentMessage.getDisplayOriginatingAddress());
25 sb.append("\n----Message----\n");
26 sb.append(currentMessage.getDisplayMessageBody());
27 }
28
29 Toast.makeText(context, sb.toString(), Toast.LENGTH_LONG).show();
30
31 }
32
33 }
34
35 }

The above code is fairly self-explanatory. You can download the
complete Eclipse project, which is based on the HelloAndroid example in the
Android Dev Guide from HERE.



Read more...

Grails Acegi ACL Howto

Acegi ACLs are a daunting subject. However I have recently integrated Acegi ACL into the Grails Acegi Plugin, and with that made it much easier to implement fine-grained ACLs in your Grails projects.

Prerequisites
Please refer to the Resources section at the end of this article for downloads


There are primarily two use-cases for the current Acegi Plugin's ACL extensions:
  1. Protecting Method Invocations
  2. Protecting Domain object instances

1. Protecting Method Invocations

You can protect method invocations by requiring logged in user to be in certain roles(ROLE_xxx). This check ignores all method parameters, and will throw an exception if the current user is not in the required Role.
e.g. method signature:
   1:public ReportLog getReportLog(Integer reportId){
2: return Report.get(reportId).log
3:}
2. Protecting domain object instances

You can protect domain object instances (tuples in database) using a variety of fine-grained access controls. Currently Acegi supports three (3) distinct methods out of the box, two (2) of which are currently directly supported by the Grails plugin.

2.1. Applying ACLs to method parameters (CURRENTLY UNSUPPORTED!)

This ACL checks the first parameter of a method for a matching domain class to see if current user has ACL on that object instance (ACL_ENTRY).
e.g. method signature:
   1:public void saveReport(Report report){
2: report.save()
3:}
will result in a check being done on the Report parameter to saveReport()
to see if the current user has an ACL granting permission for the domain object
instance in the parameter.


2.2 Filtering collections using ACLs

You can filter the domain objects in a collection returned from a method to see if the user has appropriate ACLs on each object instance in the collection. Items in collection to which the current user does not have access to are automatically removed from the list. (AFTER_ACL_COLLECTION_READ)
e.g. method signature:
   1:public List getAllReports(){
2: return Report.list()
3:}
4:
2.2. Applying ACLs to method return value

You can check the domain object returned from a method invocation to see if the current user as an appropriate ACL granting permission to that domain object. (AFTER_ACL_READ)
e.g. method signature:
   1:public Report getReport(Integer id){
2: return Report.get(id)
3:}
4:

Protecting a Service Class
Let us now look at a service class which has ACL protection turned on:
   1:class ReportService{
2:
3: static acegiACL = {
4: Report{
5: getReportLog(['ROLE_USER', 'ROLE_ADMIN'])
6: getAllReports(['ROLE_USER', 'AFTER_ACL_COLLECTION_READ'])
7: getReport(['ROLE_USER', 'AFTER_ACL_READ'])
8: saveReport(['ACL_REPORT_WRITE']) //CURRENTLY UNSUPPORTED!
9: }
10: }
11:
12: boolean transactional = true
13:
14: def getReport(Integer reportId) {
15: return Report.get( reportId )
16: }
17:
18: def getAllReports(){
19: return Report.list()
20: }
21:
22: def getReportLog(Integer reportId){
23: return Report.get(reportId).log
24: }
25:
26: def saveReport(Report report){
27: report.save()
28: }
29:
30:}
Note that there is a static closure defined called 'acegiACL'. This naming is important, and each service class you wish to protect must have this closure definition.
NOTE: The 'acegiACL' closure is not currently supported on domain objects.

The 'acegiACL' closure can be explained as follows:
  • line 3 - "Report {... "
    • refers to the domain object we are protecting. This name is used when applying ACL controls for AFTER_ACL_READ and AFTER_ACL_COLLECTION_READ.
  • line 5 - getReportLog(['ROLE_USER', 'ROLE_ADMIN'])
    • Only allow users in the ROLE_USER and ROLE_ADMIN roles to call the getReportLog() method
  • line 6 - getAllReports(['ROLE_USER', 'AFTER_ACL_COLLECTION_READ'])
    • Only allow users in the ROLE_USER role to call the getAllReports() method.
    • Additionally filter all returned results checking that user has READ permission on the domain objects.
  • line 7 - getReport(['ROLE_USER', 'AFTER_ACL_READ'])
    • Only allow users in the ROLE_USER role to call this method.
    • Additionally check the returned domain instance to see if the current user has the READ permission on the object.
  • line 8 - saveReport(['ACL_REPORT_WRITE']) - (UNSUPPORTED!)
    • Check the custom permission to verify that the current user has WRITE permission on domain object instance passed into the method.
Future improvements
  • Allow fine-grained ACLs of the type ACL_REPORT_SAVE, by automatically creating instances of AclEntryVoter
  • Create a mechanism allowing users to easily hook up their own custom instances of AbstractAclVoter into the ACL mechanism
Acknowledgments: Thanks to Tsuyoshi Yamamoto San, and Haotian Sun for their excellent initial work on the Grails Acegi Plugin

Resources:
  • Acegi Security - http://www.acegisecurity.org/
  • Download example application demonstrating ACL integration here.
  • Download development version of Grails Acegi Plugin with ACL here.
    Update: [7May2008] - This plugin download is NOT the same as the 0.3 version of the plugin available for download from grails.codehaus.org!


Read more...