First up: A new regional service provider for an association of professionals. I have nearly a dozen clients whose websites depend on a datafeed, which was provided by the previous service provider for this association. The old datafeed was excellent — clear and concise, comprehensive, and easy to use. The new provider first hemmed and hawed and offered vague responses to direct questions….. Somewhat important questions such as, “Will you still provide us with a datafeed?” If this service provider was not going to provide a datafeed, I would have to rewrite all the programming on my clients’ sites to use a different datafeed from a secondary source, which wasn’t as good as the one we were currently getting, but at least it was something. If the new provider would be providing a datafeed, I needed the schema for it as soon as possible, because the old datafeed was going to cease on a particular date, and any new datafeed would likely require rewriting my programming.

The new provider, though, simply couldn’t seem to tell us if they would be providing a datafeed or not. First the answer was “yes,” then it was “no,” then it was “maybe,” then it was “yes, and it will be compliant with the national standard,” then it was “if you pay us for the custom programming we will.” I was just about at the point where I would have to use the other available datafeed in order to get my clients’ sites updated by the deadline, when this new provider made a datafeed available.

And when I finally got a look at it: Oh boy, does it suck. First of all, and I’m actually very glad of this, the datafeed omits some of the crucial information that my clients need. I couldn’t believe that a large professional service provider such as this one would omit such critical information from the datafeed they provide to their members. Keep in mind, the members, including my clients, provide the data in the first place, and pay the service provider handsomely to maintain it for them. This isn’t some free service where you get what you pay for.

As I said, I was glad they omitted this information, because it allowed me to recommend to my clients that we use the secondary datafeed from a different provider. My clients wouldn’t have understood, or cared, about my other objections to the datafeed, but the missing information they understood and cared about.

And what are my other objections to this new feed? Well, first, it completely ignores the flexibility of relational databases, and has never even been in the same room with the concept of normalization. As an example, for a particular feature, the datafeed has 5 fields for characteristics of that feature. What will happen when 6 characteristics are needed? The datafeed will have to be restructured to add a 6th field, and my database that uses the datafeed will have to be restructured, and the programming code that selects the data and displays it in a web browser would have to be re-written (again) to accommodate the 6th characteristic. And then a 7th characteristic will appear at some point, and I’d have to go through that pointless exercise all over again. The datafeed is structured this way for about 9 different features — and it’s almost certain that these will need to be changed on a regular basis.

It also means that for a feature, say, “surface,” a particular characteristic, say “fuzzy,” might appear in field #10, or field #11, or field #12, or field #13, or field #14…. Well, you probably get the idea. There’s no consistency at all with respect to what goes where in the database. So that if I wanted to search the database for items that have a fuzzy surface, I have to write the code to search multiple different fields. This is both stupid, and highly inefficient.

The previous service provider understood relational databases and had structured their datafeed so that they could add a 6th characteristic, and a 7th, and even an 8th, 9th, and 10th, and it would continue working with my database “as is,” with no re-programming required. It’s not that I’m lazy — when re-programming is required, I do it. But I have better things to do with my time than waste it re-writing code that shouldn’t need to be rewritten, and my clients have better things to do with their money than pay me for that time.

And they did silly things like break up a single 1,200-character text field into 5 separate text fields, breaking up the text between words in a sentence, if needed, to fit them into the shorter text fields. That’s easy enough to handle in my code — I would simply concatenate all 5 fields into 1 in my database query. But it shouldn’t be necessary

The complete lack of normalization, and the moronic handling of the large text field, are indicative of a programmer who doesn’t know jack shit about databases. And that was my biggest concern of all. Even if I were to use the database as provided, what would they do with it down the road? They might decide to remove or add fields without telling anyone, or make other changes that would totally break the sites that depend on the datafeed.

We ended up using the secondary provider’s datafeed. It’s not as good as the previous one we were getting, but it’s infinitely better than the datafeed from the new provider.

Second: eBay, of all people! You would think a company the size of eBay would have highly competent programmers who know what they’re doing.

News flash: They don’t.

I have a number of sites that earn affiliate commissions from eBay’s Partner Network program. I can view reports on my earnings within eBay’s ePN website. I can also download those reports, for importing into a spreadsheet or database.

I want to be able to view and manipulate these reports in different ways than the ePN interface allows, so I’ve been working on a program that will import the downloaded reports into a database and allow me to search, sort, and analyze in a variety of ways. I think it’s a very cool program, and I may develop it further in order to release it publicly.

But the transaction reports that I get from eBay made my shake me head. One day, the report might contain 9 fields, and the next day it might have 11, and on another day it might have 13. I think there are some other possible permutations possible, too, but until I get such a report in my account I won’t know exactly how many fields it will have or in what order.

This makes it much more difficult to import the data into my database. I can’t just “LOAD DATA INFILE,” nor can I simply parse the files and do a standard INSERT query. Instead, I have to parse the files and check for the number of fields, and check to see what headings the fields have, and write several different MySQL queries to use, depending on the results of the parsing.

These are basic tab-delimited files. eBay’s programmers should know enough to just put a zero or a null value, along with the tab, for the fields that aren’t needed for a given report — such that all downloaded reports would have exactly 13 fields, always, and they would all have the same fields, in the same order, always.

I sometimes feel overwhelmed and outclassed on the programming forums where I hang out. There are so many extremely competent, highly skilled programmers who can toss off a complex INNER JOIN of 5 different tables as easily as I can type my name.

These programmers would laugh their asses off if they were to see these datafeeds and reports that I’m dealing with.

read more | digg story

Check out a screenshot: (click to enlarge)

I’ll still be using Commission Junction for some other affiliate programs I use, but eBay has gone in-house, and of course I’ve gone with ‘em.

I wrote a little php script to read eBay’s “transaction download” file and give me clickable links to the eBay items sold. I’m giving that script away for free (at least for now). You can download my ePN Report Tracker for free and run it on your own server — no need to worry about giving up your transaction report data to some anonymous stranger on the internet.

If you like my little script, please say “thank you” by giving me a link.

I figured what the heck, I’ll try it out. I had used Commission Junction for my affiliate marketing attempts, but had never had much success. It seemed that no matter what approach I used, it just didn’t result in a lot of sales. I tried targeted product ads on pages that talked about that specific product. I tried banner ads. I tried product listings from CJ’s datafeed. I tried it with photos of the product, with text links within the copy, with and without listing the price. I only got the occasional commission, and AdSense continued to dominate my passive income streams.

Almost immediately upon launching my first BANS site, I started getting commissions. My commission income is now at about one-third of my AdSense income, but it’s growing, and I expect that it may well exceed my AdSense income within the next few months.

The above graph shows my CJ earnings trend. My first affiliate commission was last May. Earnings stayed pretty moribund throughout 2007. It was in mid-December of 2007 that I bought BANS. It was in January that I started building BANS sites in earnest. You can see the progress I’ve made with affiliate commissions since starting to work with BANS.

I love that the script comes as full php source code — I’ve spent a lot of time customizing and modifying my BANS script so that it looks and behaves more the way I want it to. Some of the modifications I’ve done include:

Code Output/CSS/HTML

I edited the core files to remove the stupid non-breaking spaces, hard-coded table widths and table cell widths, and various other presentational code that was hard-coded into the script files.

The navigation is automatically populated with eBay’s categories and subcategories, and the navigation elements are indented by one, two, three or more non-breaking spaces. I removed those and added specific classes to each level of indent, along with specific ids for each level when the link is the current page. This allows me to target each category level with my css — I generally make the top-level categories bold-face and in fairly large type, then the sub-categories indented by a few pixels, and non-bold face but still in large type, then the sub-sub-categories indented by more pixels, non-bold-face and small type, and so on.

Also, by removing the hard-coded table and table cell widths, and adding specific css classes to each table cell, I can target those easily with my css as well. I’ve done this with the product listings, the search form, the site navigation, the “view more on ebay,” the pagination — pretty much all the code that’s output by the script. This gives me much greater flexibility in targeting these page elements with my css, which is very important to me, as I don’t use any of the boring default templates that come with a BANS purchase.

Call To Action

I edited the core source files to add a line of text below the item name, which reads either “Buy now,” “Bid on this item now,” or “Buy or bid now,” depending on whether the item is BIN only, auction only, or both. That line of text is a clickable link that takes the user to the product on eBay, just like the item name. This is the classic marketer’s “call to action.” You must tell the visitor what you want them to do. I’ve seen my click-through rate to eBay go way up since implementing this little hack.

I also edited the source files to make the Buy It Now button a clickable link. It struck me, whenever I’m looking at one of my own BANS sites, that I expect to be able to click that Buy It Now button, so probably other people do too. Let ‘em click it! Get that cookie planted!

Search Form

I edited the source files to automatically populate the search box with the user’s search query after they conduct a search. If someone searches for, say, “Masonic rings” they might want to refine their query to search for “Masonic gold rings.” I say, make it easy for them. Along with their search results, I give them a search box that contains their already-searched-for term so that they can easily refine that search.

View More Items on eBay

There’s a line automatically displayed at the bottom of each listing page that says “View more items on eBay.” This line is a link that takes you to the appropriate category on eBay, with your custom query already applied. This is a nice touch. I edited that line so that instead of the default “view more items on eBay” text, it reads “Visit eBay to see more [product name].” For example, on my Knife site, on the page for Santoku knives, below the listing of Santoku knives it reads “Visit eBay to see more Santoku Knives.”

Fixing Stupid Listing Formatting

I noticed that a lot of eBay sellers write item titles like this:

Product—feature—feature—feature—feature—feature

or this:

Product~feature~feature~feature~feature~feature~feature

This approach forces the product name to not wrap, which forces the listing to take up way too much space horizontally. I design my templates so that this doesn’t kill my page layout, but it does force a page width that’s way too wide for users on smaller monitors.

I used php’s simple string replacement function to replace instanced of — or ~ with space – space or space ~ space. This allows the product name to wrap appropriately.

Fixing BANS Quirks

I also added php’s mysql_real_escape_string in appropriate spots in the source code to fix a few bugs that shouldn’t exist. It’s impossible, for example, with the default BANS script, to use apostrophes in the page content, or to use quotation marks around a custom search query, or to use apostrophes in the link name for a category page.

The BANS programmers did such a bang-up job with this script, I’m astonished that they didn’t use the escape string everywhere that apostrophes and quotation marks might need to be inserted into the database. But they didn’t. Fortunately, with source code in hand, I’m able to fix that myself.

Custom Templates

I use my own template designs — none of that default stuff for me. I generally rotate among 4 or 5 different standard templates, which I’ve designed specifically to make it easy for me to place a product-related photo in the header. My templates use different color schemes, and I generally just choose whatever color scheme seems most appropriate for the particular niche I’m targeting.

I use a detailed stylesheet to style the navigation links, item listings, and other items just as I want them to appear. I generally remove the PayPal logo from the product listings, place the search box at the bottom of the page instead of the top, and repeat the navigation in the footer. Since I applied special css classes to my navigation links, I can target the main categories, sub-categories, and sub-sub-categories differently in my footer than in my header. I’ve been experimenting with showing only the main categories in the footer, by using display:none for the sub-categories in my css for the footer nav links.

Easy Privacy Policy

With the requirement of a privacy policy for all sites that use CJ and AdSense, I made that easy, too. I wrote a generic privacy policy, which lives on one of my websites, and I have a very basic, very easy page on each site that simply pulls in that page as a php include. It’s linked to from my footer. It exists as an actual page, so it automatically gets included in each site when I upload my files — I don’t have to mess around in the BANS admin area creating a new content page every time. It’s just automatically there. And by including the actual policy, if I ever need to rewrite the policy, I can re-write it once and have it updated on every single site. I love timesaving techniques.