Archive for the 'OpenZAP' Category

Braindump

Project: LibISDN ASN.1 En-/Decoder

Motivation: Handle Q.932 ROSE RPC calls.

Status: Early code, decoding mostly works, parts have to be rewritten for encoding to work (in a usable way).

Target Milestone: Being able to handle incoming MWI request from my ISDN phone and sent an answer that will make the icon on its LC-Display light up.

Project: libRTCP

Motivation: En-/Decoding of RTCP messages.

Status: Early version, can already decode some RTCP messages, needs more work.

Target Milestone: Working en-/decoding of XR and most other RTCP messages.

Project: mod_netlink

Motivation: Receive netlink events from linux kernel about IP-Address and HW-Configuration changes and inject them into the FreeSWITCH core.

Status: Some proof-of-concept code written (Binding to netlink multicast groups and receiving events + dumping them in hex+ascii / text)

Notes: Will need linux-2.6.30+ (unless you really want to run FreeSWITCH as root).

Target Milestone: Turn PoC code into mod_netlink.c, handle incoming network interface + ip address change events, handle incoming CPU up/down events, turn them into FreeSWITCH events.

Project: mod_http_rpc

Motivation: Current mod_xml_rpc sucks.

Status: Basic HTTP access works, using embedded mongoose.

Target Milestone: HTTP/XML-RPC API calls working, authentication via user directory.

Project: RTP Acceleration

Motivation: 10k+ calls on a single box, without ending up in context-switch hell caused by RTP recv/send syscalls.

Status: Working on early proof-of-concept code (userspace UDP/IP stack).

Notes: New test hardware: Managed 8-port 1GigE (HP 1800-8G, ~11.6Mpps), Dual-Port PCIe Intel 1000/PT, two Broadcom NetXtreme BCM5721 PCIe cards. Full setup will be completed after dead PSU has been replaced in test01.

Next milestone: Get UDP recv+send and ARP working in PoC, adapt code to lastest TX-Ring changes in 2.6.31+, start benchmarking.

Project: Improve FreeSWITCH startup time by starting modules in parallel

Motivation: Spend less time loading and initializing modules, some of which may block for some time in their startup function. Track inter-module dependencies and parallelize loading using a set/pool of threads executing the startup routines.

Status: Planning

Project: Improve configuration option handling of OpenZAP modules

Motivation: The current situation sucks completely

Status: Some option handling code written, needs to be integrated into OpenZAP (+ modified to fit)

Next milestone: Try to integrate this into OpenZAP, using mod_isdn

Project: Complete clean rewrite of the FreeSWITCH buildsystem using GNU Make, current Autools, system libs and support for the FHS.

Motivation: Meh… (we already depend on GNU Make anyway and some of us are able to roll their own controlled environment)

Status: Already started this some time ago, need to find a way to create a GIT repo that only contains and syncs the source code parts.

Target Milestone: Build core and most important modules using system libs, maybe add some of that fancy “make mod_foo” stuff via a makefile include that adds the neccessary rules. Add proper pkg-config support for the core (and remove fsxs).

Project: New work desk.

Motivation: Totally sucks to have no space to put stuff on (one of the reasons there’s not much progress in LibISDN development right now)

Status: Need to get vacation first (*sigh*)

libisdn: ASN.1

This has been on my ToDo list for a while now. Decoding works so far, but i’ll have to change things a bit to get encoding working too. Sample output:

successfully decoded ASN.1 BER data
duration 0 seconds, 102419 nanoseconds

===================================== ASN.1 =====================================

context specific element, tag: 1
{
        integer:        757
        oid:            0.4.0.745.1.3
        sequence
        {
                context specific element, tag: 1
                {
                        context specific element, tag: 0, size: 4 octets
                              [ 31 30 30 31 ]
                }
                context specific element, tag: 2
                {
                        enumerated:     0
                }
                context specific element, tag: 3
                {
                        integer:        0
                }
                context specific element, tag: 4
                {
                        context specific element, tag: 0, size: 4 octets
                              [ 31 30 30 31 ]
                }
                context specific element, tag: 6
                {
                        sequence
                        {
                                integer:        0
                                enumerated:     0
                        }
                }
        }
}

---------------------------------------------------------------------------------

This is an MWI Indication Invocation call sent from the ISDN phone on my desk to the OpenZAP NT it is connected to.

libisdn: call information dumping

Still have to change a couple of things here to get it working with the “api” interface of the OpenZAP module, but this is what it looks like right now:

API CALL [oz(isdn dump 1 calls)] output:
+OK call information dumped to log      

2009-07-07 23:55:22.500881 [DEBUG] Span:1 Information of all active calls:
==================================== Calldump ===================================
* CRV:  76 (0x76)                                                                
        Global call:   No                                                        
        Direction:     Inbound                                                   
        State:         258 'Overlap sending'
        L4 User data:  0x24632e0
        Timer:         T302, timeout in: 14372 ms, expire count: 0
        Events queued: 0
-----------------------------------------------------------------[  1 call(s) ]--

Command to get this output (debug loglevel!): oz isdn dump <span id> calls


libisdn: advanced error reporting, part 1

Just a short update on libisdn (the current name of the OpenZAP ISDN stack), before i fall asleep

This is how IE errors will be reported as of now, if you’ve turned on debug loglevel (i’m only abusing ERR for the pretty red color):

2009-07-07 01:02:25.848853 [ERR] Span:1 Message parser error(s)
=================== Errors reported for message -   5 (0x05) ===================
* IE id:  24 (0x18), name: "N/A", size:   3, offset:   9

  [01]  error message: Size invalid
        diagnostic:    Message element is either too small or too large
        details:       expected size: 3 - 0 octet(s)
        octets:
                |-{9}-> 18 01 83

---------------------------------------------------------------[  1 error(s) ]--

I’m going to add some more information (IE / Message names / Call info) in the future. Next step is to change parts of the message decoding function to relax some of the IE checks or, if possible, skip decoding a faulty IE so we can still check them all and report multiple issues in one report. Another thing we can add now is diagnostic information in Cause IEs sent to the other side (e.g. ID of the invalid IE in a message).

UPDATE: Some minor modifications to the output format:

2009-07-07 21:52:55.38692 [DEBUG] Span:1 Q931Umes[Q.931 NT][5]() returned -3002
2009-07-07 21:52:55.38692 [ERR] Span:1 Message parser error(s)
=================== Errors reported for message -   5 (0x05) ===================
* IE id:   4 (0x04), name: "N/A", size:   5, offset:   4

  <01>  error message: Not allowed here
        description:   Message element is not allowed in this state/message
        details:       N/A
        raw octets:
              [ 04 03 80 90 a3 ]

---------------------------------------------------------------[  1 error(s) ]--

Achievement of the day:

openzap overview

(Click to view in full scale)

OpenZAP ISDN: Past, Present and (possible) Future

Well, i had originally planned to make this a long entry about the history of the OpenZAP ISDN stack, what i’ve changed in the past months and how i think it should look and work like in the future, but…

… life sucks and stupid problems ususally come up at the worst time possible.

Let’s start with a short introduction to OpenZAP’s past:

The original OpenZAP ISDN has been written by Case Labs Ltd. and released under a BSD License. It contains both, Q.921 and Q.931 parts, that are loosely coupled. The Q.931 part has been designed to support multiple dialects (e.g. 5ESS, DMS etc.), which is a good thing, since OpenZAP is used by people in different countries and continents.

The old stack “works”… most of the time… well, unless something slighty unexpected happens…

It lacks proper stateful handling of messages, error handling, timers, the higher level API used in ozmod_isdn.c is message-centric instead of call-centric…. in short:

It’s a big mess.

The present:

I started in summer 2008 by rewriting most of the Q.921 layer based on the ITU-T specs (to add BRI support). That work is 80% complete, there are still a couple of things that have to be adressed e.g. error handling (FRMR etc.), resending of frames and sending/receiving events from/to Q.931 (Link activation and deactivation events for example). The initial goal however has been reached, BRI support works (Point-to-Point and Point-to-Multipoint mode) and the Q.921 layer is stateful and has working timers, which is a big improvement compared to what was there before i started.

Of course, if you start working on one part of the stack, you’ll have to touch the other one too…

… so i continued my rampage through the OpenZAP ISDN stack by trying to rewrite the Q.931 stack internals and its API. The current state of that work is available in the so called “api” branch of my private OpenZAP GIT repository on oss.axsentis.de (here is a link to the gitweb running on the server). There are already a lot of changes:

  • A ton of the message decoding/encoding functions have been replaced by a pair of generic ones.
  • The whole thing is stateful, which solves a lot of problems (like some stupid hacks to create a proper response to STATUS ENQUIRY out of the OpenZAP channel states (urgh)).
  • Initial support for timers is in the works (depends on some other bits, but i’ve made some progress this weekend)
  • Proper checking of messages:  Is this message valid in the current call state? Are those Information Elements (IEs) allowed? Are some mandatory ones missing?  –  A lot of that has been added (still needs some work though and testing)
  • API rework:   Slowly making some progress, there’s some initial code to send and queue events to the higher layer now (only for the Q.931 NT dialect right now).

Future:

So, where are we heading?

Let’s start with the high-level api part. The current API works by passing Q.931 messages to the API, which is nice and allows for some *cough* flexibility, but makes life harder on the upper level (creating custom messages piece by piece, keeping track of OpenZAP channel <-> Q.931 call pairs via their CRV (Call Reference Value) etc.). We’d like to retain some of that freedom (sending custom messages) but get rid of most of the manual plumbing by replacing it with a call-centric API. The goal here is being abled to do something like:

struct Q931_Call *call = Q931CallNew(trunk);
/* Add all the details like, Caller ID, Destination number etc.
 * (we'd still need to find a sane way to do that...)
 */

/* Associate the call with our higher level handle */
Q931CallSetPrivate(call, ourhandle);

/* Make the call */
Q931CallSend(call);   /* or something like that */

If you really want to know what this looks like in the old code, ozmod_isdn.c:isdn_request_channel() has it all (pretty, isn’t it?).

I’m taking a little shortcut for now (it’s getting late).

To understand the codebase a little, the most important names:

  • Q931call.c – Call specific functions, timers, lookup by CRV etc.
  • Q931mes.c – Message handling functions, generic message unpack / pack functions (Q931Umes_Generic, Q931Pmes_Generic), Table with Message <-> allowed IE lists for the (= base) Q.931 dialect.
  • Q931State*.c –  Q.931 TE / NT mode Dialect initialization function, Message processing functions and timer callbacks. Either mode is handled as a separate dialect!
  • Q931ie.c  –  Information Element pack / unpack functions (Q931Pie_* and Q931Uie_*).
  • Q931api.c – Misc (API) functions
  • Q931.c – Input / Output routines (Q931Rx* Q931Tx*, Note: Q931Rx43 means: Message coming from Layer 4 into Layer 3 (this is what gets called to send messages from ozmod_isdn.c to the Q.931 stack);   Q931Tx34: Send message to Layer 4), Logging functions, some Dummy functions (that should be moved somewhere else), also things like Q931Umes() : Wrapper functions; Initialization (and tons of other stuff).
  • Q931dialect.c – Dialect handling.
  • 5ESS/* –  5ESS Dialect (work-in-progress, started porting this one over to find parts of the dialect and message handling stuff that need refinement to support other dialects).

to be continued…

Lazyness…

I’ve been slacking a bit over the last week, working only on some smaller bits and pieces (e.g. the axdhcp2 lua backend).

My BRI test setup is finally back up (this time connected to the desktop box where i’m usually sitting at) and the last round of larger changes don’t seem to have caused a lot of problems.

So what’s next in OpenZAP land?:

There are some other fairly low-hanging fruit right now:

  • Move unexpected (invalid) message handling from Q931Rx23() / Q931Rx43() into Q931ProcUnexpectedMessage(), so each dialect can have its own version (if it needs to)
  • Change all Q931Proc* callbacks to work on call handles (getting rid of a CRV->call lookup in each callback, saving a few more lines of code and making message handling a little faster)
  • Handle Q931L4HeaderSpace and Q931L2HeaderSpace in one place instead of each and every function that uses the message buffer(s) (and perhaps makes those two values per-trunk)

Timers… turns out to need a bit more work. The basic framework is there, message handling (resending, indications to upper layer / call control) is still missing though, so those callbacks don’t really do anything useful at the moment. I know a couple of people are waiting for this and i’ll probably try to get call tear-down working, as a short-term solution, to make sure calls are getting properly disconnected if something goes terribly wrong (which is still a major improvement, compared to the old code).

For the long-term solution there a couple of problems that need to be resolved.

Message resending needs some way to keep track of older messages, either by storing them in a per-call buffer, or storing the relevant IEs that are needed to regenerate the message from scratch.

Sending indications/events to upper layers is another problem (this includes events caused by timers, e.g. a call disconnect), the reason is fairly simple: code run in the event callback may change the state of the current call, which is bad if we change the state again after the callback returns, or try to do more work depending on the call’s state. The right solution (i hope) is probably to defer calling the event/indication callback till all the important work has been done, either before a response is being sent or after that (still not entirely sure on this).

Those two issues, together with the (inevitable) neccessary api changes for the new event callback system still need some experimentation to find a good solution (and will probably cause some breakage).

Let’s see what’s going to happen in the next few weeks…

Back to coding…

The kitchen is finally usable (but not finished), time to get back to my beloved Q.931 stack.

Today’s changes include:

Q.931 Timers part #1: per timer, per mode timeout callbacks (only printing a log message for now, the actual logic will be added later).

The way the call reference is passed to the timeout callbacks has been changed to use struct Q931_Call * instead of the call index.

Some legacy code has been removed (Q931StartTimer() / -StopTimer() and the old CRV handling functions (except Q931ReleaseCRV(), which is still needed by ozmod_isdn.c)).

Next step is to get a box for testing ready and fix the fallout of the dialect rework, before i start changing and adding more code…

OpenZAP status update 2008/10

BRI support in OpenZAP trunk currently “works” (the basic stuff, it’s far from complete) with “Bristuffed” Zaptel. The ISDN library needs a lot of attention atm and most of the work happens behind the scenes in the api branch of my personal OpenZAP git tree. It’s basically a complete overhaul of the Q.931 (Layer 3) ISDN code, lot’s of cleanups, a ton of message handling functions getting replaced by a couple of generic ones, stateful handling of calls, error handling, etc. Most of the changes are “under the hood”, meaning they don’t affect the application interface of the library, so we could merge the patch into trunk, after things stabilize a little and some missing features have been added back (e.g. all of the dialects, except the basic Q.931 one are broken atm).

One of the ultimate (long-term) goals however, is to get a new layer 4 API (the part that is used in ozmod_isdn.c), that gets rid of the message-centric way of handling calls. The new API will be call-centric (well, there will still be a way to send custom messages to the stack), since message mangling is the job of layer 3, not the upper layer’s. Changing this will help simplify and improve the code in ozmod_isdn.c, so it will be easier to add new features that are currently missing.

Short-term goals, that will be adressed in the near future (read: once i get my test setup back up and running): Further improving the reworked dialect handling, adding all the (now disabled / broken) dialects back (note to self: need to find victims to test ’em), Q.931 timers (yah…), complete error handling and other random bits i forgot.

To be continued…