Monday, 17 July 2017

XSD DataSets and ignoring foreign keys

Question: I have a pretty standard table set-up in a current application using the .NET XSD DataSet and TableAdapter features. My contracts table consists of some standard contract information, with a column for the primary department. This column is a foreign key to my Departments table, where I store the basic department name, id, notes. This is all setup and functioning in my SQL Server.
When I use the XSD tool, I can drag both tables in at once and it auto detects/creates the foreign key I have between these two tables. This works great when I'm on my main page and am viewing contract data.
However, when I go to my administrative page to modify the department data. I typically do something like this:

Dim dtDepartment As New DepartmentDataTable()
Dim taDepartment As New DepartmentTableAdapter()

taDepartment.Fill(dtDepartment)

However, at this point an exception is thrown saying to the effect that there is a foreign key reference broken here, I'm guessing since I don't have the Contract DataTable filled.
How can I fix this problem? I know I can simply remove the foreign key from the XSD to make things work fine, but having the additional integrity check there and having the XSD schema match the SQL schema in the database is nice.

Solution:  You can try turning Check-constraints off on the DataSet (it's in its properties), or altering the properties of that relationship, and change the key to a simple reference - up to you.

How to export data from SQL Server 2005 to MySQL

Question: I've been banging my head against SQL Server 2005 trying to get a lot of data out. I've been given a database with nearly 300 tables in it and I need to turn this into a MySQL database. My first call was to use bcp but unfortunately it doesn't produce valid CSV - strings aren't encapsulated, so you can't deal with any row that has a string with a comma in it (or whatever you use as a delimiter) and I would still have to hand write all of the create table statements, as obviously CSV doesn't tell you anything about the data types.
What would be better is if there was some tool that could connect to both SQL Server and MySQL, then do a copy. You lose views, stored procedures, trigger, etc, but it isn't hard to copy a table that only uses base types from one DB to another... is it?
Does anybody know of such a tool? I don't mind how many assumptions it makes or what simplifications occur, as long as it supports integer, float, datetime and string. I have to do a lot of pruning, normalising, etc. anyway so I don't care about keeping keys, relationships or anything like that, but I need the initial set of data in fast!

Solution: Using MSSQL Management Studio i've transitioned tables with the MySQL OLE DB. Right click on your database and go to "Tasks->Export Data" from there you can specify a MsSQL OLE DB source, the MySQL OLE DB source and create the column mappings between the two data sources.
You'll most likely want to setup the database and tables in advance on the MySQL destination (the export will want to create the tables automatically, but this often results in failure). You can quickly create the tables in MySQL using the "Tasks->Generate Scripts" by right clicking on the database. Once your creation scripts are generated you'll need to step through and search/replace for keywords and types that exist in MSSQL to MYSQL.
Of course you could also backup the database like normal and find a utility which will restore the MSSQL backup on MYSQL. I'm not sure if one exists however.

How would you access Object properties from within an object method?

Question: What is the "purist" or "correct" way to access an object's properties from within an object method that is not a getter/setter method?
I know that from outside of the object you should use a getter/setter, but from within would you just do:
Java:
String property = this.property;

PHP:
$property = $this->property;

or would you do:
Java:
String property = this.getProperty();

PHP:
$property = $this->getProperty();

I am talking about private or protected variables/properties only. When I learned OO I was taught to use getters/setters for every single property even if it was public (and actually I was told never to make any variable/property public). So, I may be starting off from a false assumption from the get go. It appears that people answering this question are maybe saying that you should have public properties and that those don't need getters and setters, which goes against what I was taught, and what I was talking about, although maybe that needs to be discussed as well. That's probably a good topic for a different question though...

Solution: This has religious war potential, but it seems to me that if you're using a getter/setter, you should use it internally as well - using both will lead to maintenance problems down the road (e.g. somebody adds code to a setter that needs to run every time that property is set, and the property is being set internally w/o that setter being called).

Sunday, 16 July 2017

Java lib or app to convert CSV to XML file?

Question: Is there an existing application or library in Java which will allow me to convert a CSV data file to XML file?
The XML tags would be provided through possibly the first row containing column headings

Solution:  OpenCsv for parsing CSV (small, simple, reliable and easy to use)
Xstream to parse/serialize XML (very very easy to use, and creating fully human readable xml) Using the same sample data as above, code would look like:

import java.io.FileReader;
import java.io.FileWriter;
import java.util.ArrayList;
import java.util.List;

import au.com.bytecode.opencsv.CSVReader;

import com.thoughtworks.xstream.XStream;

public class CsvToXml {     

    public static void main(String[] args) {

        String startFile = "./startData.csv";
        String outFile = "./outData.xml";

        try {
            CSVReader reader = new CSVReader(new FileReader(startFile));
            String[] line = null;

            String[] header = reader.readNext();

            List out = new ArrayList();

            while((line = reader.readNext())!=null){
                List<String[]> item = new ArrayList<String[]>();
                    for (int i = 0; i < header.length; i++) {
                    String[] keyVal = new String[2];
                    String string = header[i];
                    String val = line[i];
                    keyVal[0] = string;
                    keyVal[1] = val;
                    item.add(keyVal);
                }
                out.add(item);
            }

            XStream xstream = new XStream();

            xstream.toXML(out, new FileWriter(outFile,false));

        } catch (Exception e) {
            // TODO Auto-generated catch block
            e.printStackTrace();
        }
    }
}

Producing the following result: (Xstream allows very fine tuning of the result...)


<list>
  <list>
    <string-array>
      <string>string</string>
      <string>hello world</string>
    </string-array>
    <string-array>
      <string>float1</string>
      <string>1.0</string>
    </string-array>
    <string-array>
      <string>float2</string>
      <string>3.3</string>
    </string-array>
    <string-array>
      <string>integer</string>
      <string>4</string>
    </string-array>
  </list>
  <list>
    <string-array>
      <string>string</string>
      <string>goodbye world</string>
    </string-array>
    <string-array>
      <string>float1</string>
      <string>1e9</string>
    </string-array>
    <string-array>
      <string>float2</string>
      <string>-3.3</string>
    </string-array>
    <string-array>
      <string>integer</string>
      <string>45</string>
    </string-array>
  </list>
  <list>
    <string-array>
      <string>string</string>
      <string>hello again</string>
    </string-array>
    <string-array>
      <string>float1</string>
      <string>-1</string>
    </string-array>
    <string-array>
      <string>float2</string>
      <string>23.33</string>
    </string-array>
    <string-array>
      <string>integer</string>
      <string>456</string>
    </string-array>
  </list>
  <list>
    <string-array>
      <string>string</string>
      <string>hello world 3</string>
    </string-array>
    <string-array>
      <string>float1</string>
      <string>1.40</string>
    </string-array>
    <string-array>
      <string>float2</string>
      <string>34.83</string>
    </string-array>
    <string-array>
      <string>integer</string>
      <string>4999</string>
    </string-array>
  </list>
  <list>
    <string-array>
      <string>string</string>
      <string>hello 2 world</string>
    </string-array>
    <string-array>
      <string>float1</string>
      <string>9981.05</string>
    </string-array>
    <string-array>
      <string>float2</string>
      <string>43.33</string>
    </string-array>
    <string-array>
      <string>integer</string>
      <string>444</string>
    </string-array>
  </list>
</list>

ASP.NET Site Maps

Question: Has anyone got experience creating SQL-based ASP.NET site-map providers?
I've got the default XML file web.sitemap working properly with my Menu and SiteMapPath controls, but I'll need a way for the users of my site to create and modify pages dynamically.
I need to tie page viewing permissions into the standard ASP.NET membership system as well.

Solution:  Here's my fix in BuildSiteMap():

SiteMapNode node = GetSiteMapNodeFromReader(reader);
string url = node.Url;
if (url.Contains(":"))
{
    string garbage = Guid.NewGuid().ToString();  // SiteMapNode needs unique URLs
    node.Url = "~/dummy_" + garbage + ".aspx";
    AddNode(node, _root);
    node.Url = url;
}
else
{
    AddNode(node, _root);
}

SQLDependency caching is cool, but if you don't want to make a trip to the DB everytime your menu loads (to check to see if the dependency has changed) and your menus don't change very often, then why not use HttpRuntime.Cache instead?



public override SiteMapNode RootNode
{
    get
    {
        SiteMapNode temp = (SiteMapNode)HttpRuntime.Cache["SomeKeyName"];
        if (temp == null)
        {
            temp = BuildSiteMap();
            HttpRuntime.Cache.Insert("SomeKeyName", temp, null, DateTime.Now.AddHours(1), Cache.NoSlidingExpiration);
        }
        return temp;
    }
}

Decoding T-SQL CAST in C#/VB.NET

Question: Recently our site has been deluged with the resurgence of the Asprox botnet SQL injection attack. Without going into details, the attack attempts to execute SQL code by encoding the T-SQL commands in an ASCII encoded BINARY string. It looks something like this:

DECLARE%20@S%20NVARCHAR(4000);SET%20@S=CAST(0x44004500...06F007200%20AS%20NVARCHAR(4000));EXEC(@S);--

I was able to decode this in SQL, but I was a little wary of doing this since I didn't know exactly what was happening at the time.
I tried to write a simple decode tool, so I could decode this type of text without even touching SQL Server. The main part I need decoded is:
CAST(0x44004500...06F007200 AS
NVARCHAR(4000))

I've tried all of the following commands with no luck:
txtDecodedText.Text =
    System.Web.HttpUtility.UrlDecode(txtURLText.Text);
txtDecodedText.Text =
    Encoding.ASCII.GetString(Encoding.ASCII.GetBytes(txtURLText.Text));
txtDecodedText.Text =
    Encoding.Unicode.GetString(Encoding.Unicode.GetBytes(txtURLText.Text));
txtDecodedText.Text =
    Encoding.ASCII.GetString(Encoding.Unicode.GetBytes(txtURLText.Text));
txtDecodedText.Text =
    Encoding.Unicode.GetString(Convert.FromBase64String(txtURLText.Text));

What is the proper way to translate this encoding without using SQL Server? Is it possible? I'll take VB.NET code since I'm familiar with that too.

Okay, I'm sure I'm missing something here, so here's where I'm at.
Since my input is a basic string, I started with just a snippet of the encoded portion - 4445434C41 (which translates to DECLA) - and the first attempt was to do this...

txtDecodedText.Text = Encoding.UTF8.GetString(Encoding.UTF8.GetBytes(txtURL.Text));

...and all it did was return the exact same thing that I put in, since it converted each character into is byte.
I realized that I need to parse each two characters into a byte manually since I don't know of any methods yet that will do that, so now my little decoder looks something like this:

while (!boolIsDone)
{
    bytURLChar = byte.Parse(txtURLText.Text.Substring(intParseIndex, 2));
    bytURL[intURLIndex] = bytURLChar;
    intParseIndex += 2;
    intURLIndex++;

    if (txtURLText.Text.Length - intParseIndex < 2)
    {
        boolIsDone = true;
    }
}

txtDecodedText.Text = Encoding.UTF8.GetString(bytURL);

Things look good for the first couple of pairs, but then the loop balks when it gets to the "4C" pair and says that the string is in the incorrect format.
Interestingly enough, when I step through the debugger and to the GetString method on the byte array that I was able to parse up to that point, I get ",-+" as the result.
How do I figure out what I'm missing - do I need to do a "direct cast" for each byte instead of attempting to parse it?

Solution: did some more poking and realized that I did need to do a double conversion, and eventually worked out this little nugget:

Convert.ToString(Convert.ToChar(Int32.Parse(EncodedString.Substring(intParseIndex, 2), System.Globalization.NumberStyles.HexNumber)));

From there I simply made a loop to go through all the characters 2 by 2 and get them "hexified" and then translated to a string.

Anatomy of a “Memory Leak”

Question: In .NET perspective:
  • What is a Memory Leak?
  • How can you determine whether your application leaks? What are the effects?
  • How can you prevent a memory leak?
  • If your application has memory leak, does it go away when the process exits or is killed? Or do memory leaks in your application affect other processes on the system even after process completion?
  • And what about unmanaged code accessed via COM Interop and/or P/Invoke?
I have some answers for these questions myself, but they are incomplete. What do you think?

Solution: The best explanation I've seen is in Chapter 7 of the free Foundations of Programming ebook.
Basically, in .NET a memory leak occurs when referenced objects are rooted and thus cannot be garbage collected. This occurs accidentally when you hold on to references beyond the intended scope.
You'll know that you have leaks when you start getting outofmemoryexceptions or your memory usage goes up beyond what you'd expect (perfmon has nice memory counters).
Understanding .NET's memory model is your best way of avoiding it. Specifically, understanding how the garbage collector works and how references work (again, I refer you to chapter 7 of the ebook). Also, be mindful of common pitfalls, probably the most common being events. If object A registered to an event on object B, then object A will stick around until object B disappears because B holds a reference to A. The solution is to unregister your events when you're done.
Of course, a good memory profile will let you see your object graphs and explore the nesting/referencing of your objects to see where references are coming from and what root object is responsible (red-gate ants profile, JetBrains dotMemory, memprofiler are really good choices, or you can use the text-only windbg and sos, but I'd strongly recommend a commercial/visual product unless you're a real guru).
I believe unmanaged code is subject to typical memory leaks of unamanged code, except that references shared between the two are managed by the garbage collector. Could be wrong about this last point.

Saturday, 15 July 2017

Is gettimeofday() guaranteed to be of microsecond resolution?

Question: So I find myself porting a game, that was originally written for the Win32 API, to Linux (well, porting the OS X port of the Win32 port to Linux). I have implemented QueryPerformanceCounter by giving the uSeconds since the process start up:

BOOL QueryPerformanceCounter(LARGE_INTEGER* performanceCount)
{
    gettimeofday(&currentTimeVal, NULL);
    performanceCount->QuadPart = (currentTimeVal.tv_sec - startTimeVal.tv_sec);
    performanceCount->QuadPart *= (1000 * 1000);
    performanceCount->QuadPart += (currentTimeVal.tv_usec - startTimeVal.tv_usec);

    return true;
}

This, coupled with QueryPerformanceFrequency() giving a constant 1000000 as the frequency, works well on my machine, giving me a 64 bit variable that contains uSeconds since the program's start up. So is this portable? I don't want to discover it works differently if the kernel was compiled in a certain way or anything like that. I am fine with it being non-portable to something other than Linux, however.

Solution: Maybe. But you have bigger problems. gettimeofday() can result in incorrect timings if there are processes on your system that change the timer (ie, ntpd). On a "normal" linux, though, I believe the resolution of gettimeofday() is 10us. It can jump forward and backward and time, consequently, based on the processes running on your system. This effectively makes the answer to your question no.
You should look into clock_gettime(CLOCK_MONOTONIC) for timing intervals. It suffers from several less issues due to things like multi-core systems and external clock settings.
Also, look into the clock_getres() function.

Friday, 14 July 2017

Flat File Databases

Question: What are your best practices around creating flat file database structures in PHP? A lot of the more mature PHP flat file frameworks I see out there attempt to implement SQL-like query syntax, which is over the top for my purposes in most cases (I would just use a database at that point).
Are there any elegant tricks out there to get good performance and features with the small code overhead one would want by taking on this problem in the first place?

Solution:  Well, what is the nature of the flat databases. Are they large or small. Is it simple arrays with arrays in them? if its something simple say userprofiles built as such:

$user = array("name" => "dubayou", 
              "age" => 20,
              "websites" => array("dubayou.com","willwharton.com","codecream.com"),
              "and_one" => "more");

and to save or update the db record for that user.


$dir = "../userdata/";  //make sure to put it bellow what the server can reach.
file_put_contents($dir.$user['name'],serialize($user));

and to load the record for the user


function get_user($name){
    return unserialize(file_get_contents("../userdata/".$name));
}

SQLStatement.execute() - multiple queries in one statement

Question: I've written a database generation script in SQL and want to execute it in my Adobe AIR application:

Create Table tRole (
    roleID integer Primary Key
    ,roleName varchar(40)
);
Create Table tFile (
    fileID integer Primary Key
    ,fileName varchar(50)
    ,fileDescription varchar(500)
    ,thumbnailID integer
    ,fileFormatID integer
    ,categoryID integer
    ,isFavorite boolean
    ,dateAdded date
    ,globalAccessCount integer
    ,lastAccessTime date
    ,downloadComplete boolean
    ,isNew boolean
    ,isSpotlight boolean
    ,duration varchar(30)
);
Create Table tCategory (
    categoryID integer Primary Key
    ,categoryName varchar(50)
    ,parent_categoryID integer
);

I execute this in Adobe AIR using the following methods:
public static function RunSqlFromFile(fileName:String):void {
    var file:File = File.applicationDirectory.resolvePath(fileName);
    var stream:FileStream = new FileStream();
    stream.open(file, FileMode.READ)
    var strSql:String = stream.readUTFBytes(stream.bytesAvailable);
    NonQuery(strSql);
}

public static function NonQuery(strSQL:String):void {
    var sqlConnection:SQLConnection = new SQLConnection();
    sqlConnection.open(File.applicationStorageDirectory.resolvePath(DBPATH));
    var sqlStatement:SQLStatement = new SQLStatement();
    sqlStatement.text = strSQL;
    sqlStatement.sqlConnection = sqlConnection;
    try {
        sqlStatement.execute();
    } catch (error:SQLError) {
        Alert.show(error.toString());
    }
}


No errors are generated, however only tRole exists. It seems that it only looks at the first query (up to the semicolon- if I remove it, the query fails). Is there a way to call multiple queries in one statement?

 Solution: 

It is a kind of a hack, but it actually works pretty well. The only thing is you have to be very careful with your semicolons.


var strSql:String = stream.readUTFBytes(stream.bytesAvailable);      
var i:Number = 0;
var strSqlSplit:Array = strSql.split(";");
for (i = 0; i < strSqlSplit.length; i++){
    NonQuery(strSqlSplit[i].toString());
}

Thursday, 13 July 2017

Getting Subclipse in Aptana to work with the newest release of Subversion

Question: The version of Subclipse (1.2.4) currently available through Aptana's automatic Plugins Manager does not work with the newest version of Subversion.
I see on the Subclipse website however that they have 1.4.2 out for Eclipse. So I added a new remote update site to my Update manager. When I tried to install it, it told me I needed Mylyn 3.0.0. So after much searching I found Mylyn 3.0.0 and added another new remote update site to my update manager. Then when I tried to install that, it told me I needed org.eclipse.ui 3.3.0 or equivalent.
Looking at the configuration details for Aptana, it looks like it is built against eclipse 3.2.2.
Does anyone know if there is a way to upgrade the version of Eclipse Aptana that is built against to 3.3.0? Or if there is some other way to get Subclipse to work with the very newest version of Subversion?
I know this isn't necessarily a "programming" question, but I hope it's ok since it's highly relevant to the programming experience.

Solution:  Subclipse does not require Mylyn, but the update site includes a plugin that integrates Mylyn and Subclipse. This is intended for people that use Mylyn. In your case, you would want to just de-select Mylyn in the update dialog.
Subclipse also requires Subversion 1.5 and the corresponding version of the JavaHL native libraries. I have written the start of an FAQ to help people understand JavaHL and how to get it.

How do I add existing comments to RDoc in Ruby?

Question: I want to format my existing comments as 'RDoc comments' so they can be viewed using ri.
What are some recommended resources for starting out using RDoc?

Solution: A few things that have bitten me:
  • :main: -- RDoc uses only the last one evaluated; best to make sure there's only one in your project and you don't also use the --main command-line argument.
  • same as previous, but for :title:
  • :section: doesn't work very well

Paging a collection with LINQ

Question: How do you page through a collection in LINQ given that you have a startIndex and a count?

Solution:  Following natural way of paginating a LINQ collection.

var query = from i in ideas
            select i;
var pagedCollection = query.InPagesOf(10);
var pageOfIdeas = pagedCollection.Page(2);

Office 2007 File Type, Mime Types and Identifying Characters

Question: Where can I find a list of the MIME types and identifying characters for Office 2007 files?
I have an upload form that is restricting uploads based on extension and identifying characters, but I cannot seem to find the Office 2007 MIME types.

Solution:  Here are the list of MIME types and identifying characters for Office 2007 files
  • .docm, application/vnd.ms-word.document.macroEnabled.12
  • .docx, application/vnd.openxmlformats-officedocument.wordprocessingml.document
  • .dotm, application/vnd.ms-word.template.macroEnabled.12
  • .dotx, application/vnd.openxmlformats-officedocument.wordprocessingml.template
  • .potm, application/vnd.ms-powerpoint.template.macroEnabled.12
  • .potx, application/vnd.openxmlformats-officedocument.presentationml.template
  • .ppam, application/vnd.ms-powerpoint.addin.macroEnabled.12
  • .ppsm, application/vnd.ms-powerpoint.slideshow.macroEnabled.12
  • .ppsx, application/vnd.openxmlformats-officedocument.presentationml.slideshow
  • .pptm, application/vnd.ms-powerpoint.presentation.macroEnabled.12
  • .pptx, application/vnd.openxmlformats-officedocument.presentationml.presentation
  • .xlam, application/vnd.ms-excel.addin.macroEnabled.12
  • .xlsb, application/vnd.ms-excel.sheet.binary.macroEnabled.12
  • .xlsm, application/vnd.ms-excel.sheet.macroEnabled.12
  • .xlsx, application/vnd.openxmlformats-officedocument.spreadsheetml.sheet
  • .xltm, application/vnd.ms-excel.template.macroEnabled.12
  • .xltx, application/vnd.openxmlformats-officedocument.spreadsheetml.template

How do I get a distinct, ordered list of names from a DataTable using LINQ?

Question: I have a DataTable with a Name column. I want to have a collection of the unique names ordered alphabetically. The following query ignores the order by clause.
var names =
    (from DataRow dr in dataTable.Rows
    orderby (string)dr["Name"]
    select (string)dr["Name"]).Distinct();

Why does the orderby not get enforced?

Solution:  To make it more readable and maintainable, you can also split it up into multiple LINQ statements.
  1. First, select your data into a new list, let's call it x1, do a projection if desired
  2. Next, create a distinct list, from x1 into x2, using whatever distinction you require
  3. Finally, create an ordered list, from x2 into x3, sorting by whatever you desire

Wednesday, 12 July 2017

Multiple submit buttons in an HTML form

Question: Let's say you create a Wizard in an HTML form. One button goes back, and one goes forward. Since the back button appears first in the markup when you press Enter it will use that button to submit the form.
Example:

<form>
  <!-- put your cursor in this field and press Enter -->
  <input type="text" name="field1" />

  <!-- This is the button that will submit -->
  <input type="submit" name="prev" value="Previous Page" />

  <!-- But this is the button that I WANT to submit -->
  <input type="submit" name="next" value="Next Page" />
</form>

What I would like to do, is get to decide which button is used to submit the form when a user presses Enter. That way, when you press Enter the Wizard will move to the next page, not the previous. Do you have to use tabindex to do this?

Solution:

I'm just doing the trick of floating the buttons on the right.

This way the Prev button is left of the Next button but the Next comes first in the HTML code:

.f {
  float: right;
}
.clr {
  clear: both;
}

HTML
<form action="action" method="get">
  <input type="text" name="abc">
  <div id="buttons">
    <input type="submit" class="f" name="next" value="Next">
    <input type="submit" class="f" name="prev" value="Prev">
    <div class="clr"></div><!-- This div prevents later elements from floating with the buttons. Keeps them 'inside' div#buttons -->
  </div>
</form>

Tuesday, 11 July 2017

Best way to allow plugins for a PHP application

Question: I am starting a new web application in PHP and this time around I want to create something that people can extend by using a plugin interface.
How does one go about writing 'hooks' into their code so that plugins can attach to specific events?

Solution:

 You could use an Observer pattern. A simple functional way to accomplish this:


<?php

/** Plugin system **/

$listeners = array();

/* Create an entry point for plugins */
function hook() {
    global $listeners;

    $num_args = func_num_args();
    $args = func_get_args();

    if($num_args < 2)
        trigger_error("Insufficient arguments", E_USER_ERROR);

    // Hook name should always be first argument
    $hook_name = array_shift($args);

    if(!isset($listeners[$hook_name]))
        return; // No plugins have registered this hook

    foreach($listeners[$hook_name] as $func) {
        $args = $func($args); 
    }
    return $args;
}

/* Attach a function to a hook */
function add_listener($hook, $function_name) {
    global $listeners;
    $listeners[$hook][] = $function_name;
}

/////////////////////////

/** Sample Plugin **/
add_listener('a_b', 'my_plugin_func1');
add_listener('str', 'my_plugin_func2');

function my_plugin_func1($args) {
    return array(4, 5);
}

function my_plugin_func2($args) {
    return str_replace('sample', 'CRAZY', $args[0]);
}

/////////////////////////

/** Sample Application **/

$a = 1;
$b = 2;

list($a, $b) = hook('a_b', $a, $b);

$str  = "This is my sample application\n";
$str .= "$a + $b = ".($a+$b)."\n";
$str .= "$a * $b = ".($a*$b)."\n";

$str = hook('str', $str);
echo $str;
?>

Output
This is my CRAZY application
4 + 5 = 9
4 * 5 = 20

Notes: 

For this example source code, you must declare all your plugins before the actual source code that you want to be extendable. I've included an example of how to handle single or multiple values being passed to the plugin. The hardest part of this is writing the actual documentation which lists what arguments get passed to each hook. This is just one method of accomplishing a plugin system in PHP. There are better alternatives, I suggest you check out the WordPress Documentation for more information.

Reliable timer in a console application

Question: I am aware that in .NET there are three timer types (see Comparing the Timer Classes in the .NET Framework Class Library). I have chosen a threaded timer as the other types can drift if the main thread is busy, and I need this to be reliable.
The way this timer works in the control of the timer is put on another thread so it can always tick along with the work begin completed on the parent thread when it is not busy.
The issue with this timer in a console application is that while the timer is ticking along on another thread the main thread is not doing anything so the application closes.
I tried adding a while true loop, but then the main thread is too busy when the timer does go off.

Solution:

You can use something like Console.ReadLine() to block the main thread, so other background threads (like timer threads) will still work. You may also use an AutoResetEvent to block the execution, then (when you need to) you can call Set() method on that AutoResetEvent object to release the main thread. Also ensure that your reference to Timer object doesn't go out of scope and garbage collected.

Monday, 10 July 2017

Check for changes to an SQL Server table?

Question: How can I monitor an SQL Server database for changes to a table without using triggers or modifying the structure of the database in any way? My preferred programming environment is .NET and C#.
I'd like to be able to support any SQL Server 2000 SP4 or newer. My application is a bolt-on data visualization for another company's product. Our customer base is in the thousands, so I don't want to have to put in requirements that we modify the third-party vendor's table at every installation.
By "changes to a table" I mean changes to table data, not changes to table structure.
Ultimately, I would like the change to trigger an event in my application, instead of having to check for changes at an interval.

The best course of action given my requirements (no triggers or schema modification, SQL Server 2000 and 2005) seems to be to use the BINARY_CHECKSUM function in T-SQL. The way I plan to implement is this:
Every X seconds run the following query:

SELECT CHECKSUM_AGG(BINARY_CHECKSUM(*))
FROM sample_table
WITH (NOLOCK);

And compare that against the stored value. If the value has changed, go through the table row by row using the query:

SELECT row_id, BINARY_CHECKSUM(*)
FROM sample_table
WITH (NOLOCK);

And compare the returned checksums against stored values.

Solution:

 Take a look at the CHECKSUM command:

SELECT CHECKSUM_AGG(BINARY_CHECKSUM(*)) FROM sample_table WITH (NOLOCK);

That will return the same number each time it's run as long as the table contents haven't changed. 

Saturday, 8 July 2017

How to use the C socket API in C++ on z/OS

Question: I've been having issues getting the C sockets API to work properly in C++. Specifically, although I am including sys/socket.h, I still get compile time errors telling me that AF_INET is not defined. Am I missing something obvious, or could this be related to the fact that I'm doing this coding on z/OS and my problems are much more complicated?

Update: Upon further investigation, I discovered that there is an #ifdef that I'm hitting. Apparently z/OS isn't happy unless I define which "type" of sockets I'm using with:

#define _OE_SOCKETS

Now, I personally have no idea what this _OE_SOCKETS is actually for, so if any z/OS sockets programmers are out there (all 3 of you), perhaps you could give me a rundown of how this all works?

Sure I can post a test app.

#include <sys/socket.h>

int main()
{
    return AF_INET;
}

Compile/Link Output:
cxx -Wc,xplink -Wl,xplink -o inet_test inet.C
"./inet.C", line 5.16: CCN5274 (S) The name lookup for "AF_INET" did not find a declaration.
CCN0797(I) Compilation failed for file ./inet.C. Object file not created.
A check of sys/sockets.h does include the definition I need, and as far as I can tell, it is not being blocked by any #ifdef statements.
I have however noticed it contains a the following:

#ifdef __cplusplus
  extern "C" {
#endif

which encapsulates basically the whole file. Not sure if it matters.

Solution:

 X/Open

#define _XOPEN_SOURCE_EXTENDED 1
#include <sys/socket.h>

int connect(int socket, const struct sockaddr *address, socklen_t address_len);

Berkeley Sockets

#define _OE_SOCKETS
#include <sys/types.h>
#include <sys/socket.h>

int connect(int socket, struct sockaddr *address, int address_len);

Friday, 7 July 2017

Throw an error in a MySQL trigger

Question: If I have a trigger before the update on a table, how can I throw an error that prevents the update on that table?

Solution:

 As of MySQL 5.5, you can use the SIGNAL syntax to throw an exception:

signal sqlstate '45000' set message_text = 'My Error Message';

State 45000 is a generic state representing "unhandled user-defined exception".

Here is a more complete example of the approach:


delimiter //
use test//
create table trigger_test
(
    id int not null
)//
drop trigger if exists trg_trigger_test_ins //
create trigger trg_trigger_test_ins before insert on trigger_test
for each row
begin
    declare msg varchar(128);
    if new.id < 0 then
        set msg = concat('MyTriggerError: Trying to insert a negative value in trigger_test: ', cast(new.id as char));
        signal sqlstate '45000' set message_text = msg;
    end if;
end
//

delimiter ;
-- run the following as seperate statements:
insert into trigger_test values (1), (-1), (2); -- everything fails as one row is bad
select * from trigger_test;
insert into trigger_test values (1); -- succeeds as expected
insert into trigger_test values (-1); -- fails as expected
select * from trigger_test;

What is the fastest way to get the value of π?

Question: I'm looking for the fastest way to obtain the value of π, as a personal challenge. More specifically I'm using ways that don't involve using #defined constants like M_PI, or hard-coding the number in.

Solution:

Bellard's formula, as used by PiHex, the now-completed distributed computing project, is used to calculate the nth digit of π in base 2. It is a faster version (about 43% faster) of the Bailey–Borwein–Plouffe formula.


/**
 * Prints the nth number of pi followed by the next 8 numbers in base 10.
 * This program is based on Bellard's work.
 */

public class Bpp {
 
 final static int NUM = 990; // nth number of pi to print out

 /**
  * Returns the nth digit of pi followed by the next 8 numbers
  * @param n - nth number of pi to return
  * @return returns an integer value containing 8 digits after n
  */
 public int getDecimal(long n) {
  long av, a, vmax, N, num, den, k, kq, kq2, t, v, s, i;
  double sum;

  N = (long) ((n + 20) * Math.log(10) / Math.log(2));

  sum = 0;

  for (a = 3; a <= (2 * N); a = nextPrime(a)) {

   vmax = (long) (Math.log(2 * N) / Math.log(a));
   av = 1;
   for (i = 0; i < vmax; i++)
    av = av * a;

   s = 0;
   num = 1;
   den = 1;
   v = 0;
   kq = 1;
   kq2 = 1;

   for (k = 1; k <= N; k++) {

    t = k;
    if (kq >= a) {
     do {
      t = t / a;
      v--;
     } while ((t % a) == 0);
     kq = 0;
    }
    kq++;
    num = mulMod(num, t, av);

    t = (2 * k - 1);
    if (kq2 >= a) {
     if (kq2 == a) {
      do {
       t = t / a;
       v++;
      } while ((t % a) == 0);
     }
     kq2 -= a;
    }
    den = mulMod(den, t, av);
    kq2 += 2;

    if (v > 0) {
     t = modInverse(den, av);
     t = mulMod(t, num, av);
     t = mulMod(t, k, av);
     for (i = v; i < vmax; i++)
      t = mulMod(t, a, av);
     s += t;
     if (s >= av)
      s -= av;
    }

   }

   t = powMod(10, n - 1, av);
   s = mulMod(s, t, av);
   sum = (sum + (double) s / (double) av) % 1;
  }
  return (int) (sum * 1e9); // 1e9 is 9 decimal places
 }

 private long mulMod(long a, long b, long m) {
  return (long) (a * b) % m;
 }

 private long modInverse(long a, long n) {
  long i = n, v = 0, d = 1;
  while (a > 0) {
   long t = i / a, x = a;
   a = i % x;
   i = x;
   x = d;
   d = v - t * x;
   v = x;
  }
  v %= n;
  if (v < 0)
   v = (v + n) % n;
  return v;
 }

 private long powMod(long a, long b, long m) {
  long tempo;
  if (b == 0)
   tempo = 1;
  else if (b == 1)
   tempo = a;

  else {
   long temp = powMod(a, b / 2, m);
   if (b % 2 == 0)
    tempo = (temp * temp) % m;
   else
    tempo = ((temp * temp) % m) * a % m;
  }
  return tempo;
 }

 private boolean isPrime(long n) {
  if (n == 2 || n == 3)
   return true;
  if (n % 2 == 0 || n % 3 == 0 || n < 2)
   return false;

  long sqrt = (long) Math.sqrt(n) + 1;

  for (long i = 6; i <= sqrt; i += 6) {
   if (n % (i - 1) == 0)
    return false;
   else if (n % (i + 1) == 0)
    return false;
  }
  return true;
 }

 private long nextPrime(long n) {
  if (n < 2)
   return 2;
  if (n == 9223372036854775783L) {
   System.err.println("Next prime number exceeds Long.MAX_VALUE: " + Long.MAX_VALUE);
   return -1;
  }
  for (long i = n + 1;; i++)
   if (isPrime(i))
    return i;
 }

 /**
  * Runs the program
  * @param args
  */
 public static void main(String args[]) {

  long duration = System.currentTimeMillis();

  Bpp bpp = new Bpp();
  System.out.println("Decimal digits of pi at position " + NUM + ": " + bpp.getDecimal(NUM) + "\n");

  duration = System.currentTimeMillis() - duration;
  System.out.println("> " + duration + " ms");
 }

}