Setting up POP3 and SMTP Relays with Exchange 2010

Wow, recently had SO much fun struggling with Exchange 2010, to set up a POP3 connectors for external mailboxes and an SMTP relay for an internal SQL mail server. The challenge was, that when I could relay internal mail to external, my POP mailboxes fell over, and vice versa...In the end, I had to delete all the connectors Exchange created by default, and set them up manually.

It all started when I was trying to send a SQL mail using CDO. I kept getting "-2147220977" error code, which translates into "0x8004020FL - The server rejected one or more recipient addresses" (Convert the code to hex, and look at TechNet for the meaning). When I rewrote my code to C# (.Net), it popped up with "5.7.1 Unable to relay". So, at this point, I was sure SMTP was the problem.

I deleted all the SMTP connectors, and recreated one for outgoing routing, and it worked... Or so I thought... even though the mail goes out, I now could not download any mail from POP3 mailboxes. When I started the POP3 download, Event log had an error event :"Cannot connect to the SMTP server 'localhost' on port 25. The error code was 0x800ccc0f. Verify that the Microsoft Exchange Transport service is running and that the Exchange receive connectors are properly configured"



Googling did not help much, as it seems not a lot of people have had this problem before, so out of desperation, I took to my trusty whiteboard, and designed what I wanted it to do (I know, such a developer thing to do, but hey, it worked). I came up with an action plan, executed it, and lo and behold, it worked.

This is what I did...

1. In the Exchange Management Console, go to Server Configuration, Hub Transport, and delete ALL receive connectors

Now let the fun start:

1. Using the Wizard, Create a new Connector, called "POP3 Mailbox Connector". Set the intended use to "Custom" and click on next
2. At Local Network Settings, remove the default, and add specific IP 127.0.0.1
   . For the FQDN, use the FQDN of the Exchange server, and click on next - This will tell the server which server to send the incoming mail to.
3. At Remote Network Settings, remove the default, and add "127.0.0.1". This will tell the server which machine will be doing the POP3 pickups. Click on next, and run through the confirmation screens (New, Finish)

At this point the connection is set, but because of security not set, it can't do anything

4. Double click on the new connection
5. Go to Authentication, and enable Basic Authentication only. This will allow the POP connector to send credentials to the remove POP mailbox server
6. Click on Permission Groups, and set Anonymous Users and Exchange Servers. This will let the current server connect to the boxes, either through the NW Server account or through and Exchange account
7. Click on Apply, and the POP3 component is set.


Now for the SMTP relay

1. Using the Wizard, Create a new Connector, called "SMTP Relay Connector". Set the intended use to "Custom" and click on next
2. At Local Network Settings, remove the default, and add the specific IP address of the exchange server
   for the FQDN, use the FQDN of the Exchange server, and click on next - This will tell the server which server to send the incoming mail to.
3. At Remote Network Settings, remove the default, and add the IP of the server you want to send emails from. Click on next, and run through the confirmation screens (New, Finish)

At this point the connection is set, but because of security not set, it can't do anything

4. Double click on the new connection
5. Click on Permission Groups, and set Exchange Servers. This will let the current server allow connections from the sending server
6. Go to Authentication, and enable Transport Layer Security (TLS) and Externally Secured. This will allow the server to receive the internally generated emails, and get the response from external domains
7. Click on Apply, and the SMTP component is set.

Now test, and there ya go... Hopefully this will make your searching simpler than what I had to go through to get it to work :)

Great Plains Error: User is busy doing a final reconcile

We had a user who's laptop kept crashing, and it would lock him out of GP. One of the errors were "User xxx is busy doing a final reconcile. Please post later" when trying to post CB batches I found an amazing lack of info on the web re this error. So, I delved into SQL, and queried each and every table for the cashbooks I could find (CB%). I eventually found a table "CB900034" that looked like it held the locks. I deleted his record, and voila, it worked. Whether it is the right way to do it or not, I can't say, but the ends justify the means... the code I used was:

delete CB900034 where CHEKBKID = 'the cashbook id goes here'

Data Profiling Script

I recently needed to profile a database - pull out the schema, and also do some basic data quality checks. So, in the spirit of laziness, er automation, I wrote the following:

if object_Id('tempdb..#Output') is not null
drop table #output
go
Set Nocount on
go
declare @DataBase varchar(200) -- the name of the Database

Select @DataBase = 'Put Your Table Name here' -- Remember to enclose in quotes

if OBJECT_ID('tempdb..#Output') is not null
drop table #Output

select SO.name as TableName
,SC.name as ColumnName
,ST.name as DataType
,SC.length as DatTypePrecision
,CONVERT(int,0) as [RowCount]
,CONVERT(int,0) as DistinctRowCount
,CONVERT(int,0) as MinLength
,CONVERT(int,0) as MaxLength
,CONVERT(float,0) as AVGValue
,CONVERT(varchar(200),'') as MinValue
,CONVERT(varchar(200),'') as MaxValue
,CONVERT(bit,0) as isDateField
,CONVERT(bit,0) as isValidDate
into #Output
from sysObjects SO
join sysColumns SC
on SO.id = SC.id
join sys.Types ST
on SC.xType = ST.system_type_ID

where so.name = @DataBase

declare curLoop cursor
For
Select TableName
,ColumnName
,DataType
from #Output

Declare @TableName varchar(200)
,@Column Varchar(200)
,@DataType Varchar(200)
,@SQL varchar(max)
Open CurLoop

Fetch Next From curLoop
into @TableName
,@Column
,@DataType


While @@FETCH_STATUS = 0
begin

Select @SQL = 'update #Output set [RowCount]= (select count([' + @Column + ']) from [' + @TableName + ']) where [ColumnName]=''' + @Column +''''
exec (@SQL)

Select @SQL = 'update #Output set [DistinctRowCount]= (select count(distinct [' + @Column + ']) from [' + @TableName + ']) where [ColumnName]=''' + @Column +''''
exec (@SQL)

Select @SQL = 'update #Output set [minLength]= (select min(len([' + @Column + '])) from [' + @TableName + ']) where [ColumnName]=''' + @Column +''''
exec (@SQL)

Select @SQL = 'update #Output set [MaxLength]= (select max(len([' + @Column + '])) from [' + @TableName + ']) where [ColumnName]=''' + @Column +''''
exec (@SQL)

Select @SQL = 'update #Output set [AVGValue]= (select AVG([' + @Column + ']) from [' + @TableName + ']) where [DataType] in (''int'',''float'') and [ColumnName]=''' + @Column +''''
BEGIN TRY
exec (@SQL)
END TRY
BEGIN CATCH
Select @SQL = 'update #Output set [AVGValue]= 0 where [ColumnName]=''' + @Column +''''
END CATCH

Select @SQL = 'update #Output set [MinValue]= (select min(convert(varchar(max),[' + @Column + '])) from [' + @TableName + ']) where [ColumnName]=''' + @Column +''''
exec (@SQL)

Select @SQL = 'update #Output set [MaxValue]= (select max(convert(varchar(max),[' + @Column + '])) from [' + @TableName + ']) where [ColumnName]=''' + @Column +''''
exec (@SQL)


Select @SQL = 'update #Output set [isDateField]= case when columnname like (''%Date%'') then 1 else 0 end where [ColumnName]=''' + @Column +''''
exec (@SQL)
--print @SQL

Select @SQL = 'update #Output set [isValidDate]= 1 where [ColumnName]=''' + @Column +'''and [isDateField]=1 and IsDate([MinValue]) = 1 and IsDate([MaxValue]) = 1'
exec (@SQL)


Fetch Next From curLoop
into @TableName
,@Column
,@DataType

end

close curLoop
Deallocate curloop





select *
from #Output

SQL Error 18456 State 28000

Hi

I recently came accross a machine that had me confused... The connection from this machine to our production SQL server refused to work... here is what I tested:

Before I start, here are our actors:
* Machine A - The client that won't connect
* Machine B - My Dev machine
* SQL A - Our Production SQL server
* SQL B - Our dev server
* Login A - The login of the regular user of Machine A
* Login B - My admin login
* Profile A - The user's profile
* Profile B - my windows profile

So, to the test:
1. Profile A - Machine A to SQL A using Login A... Failed
2. Profile A - Machine A to SQL A using the "sa" account... Failed
3. Profile A - Machine A to SQL B... Works 100%
4. Profile B - Machine A to SQL A using Login A... Success
5. Profile B - Machine A to SQL A using the "sa" account... Failed
6. Profile B - Machine A to SQL B... Works 100%
7. Profile A - Machine B to SQL A using Login A... Works 100%
8. Profile A - Machine B to SQL A using the "sa" account... Works 100%
9. Profile A - Machine B to SQL B... Works 100%

So, the credentials are correct. It's only when you connect Machine A to SQL A that there is an error.

Lots of googling revealed a lot of people saying stuff like "Check your passwords"... NOT HELPFUL!!! At this point, if you hadn't checked the passwords, you deserve a kick in the patootie!!

So, I asked a DBA what he tought, and he suggested I check the SQL Network Client Configuration. There was an alias to the server... it was 100% correct (the right IP, and the right details), but I deleted it anyway...

LO AND BEHOLD, IT WORKED!!!

Go figure!!!

Building Excel 2007 Pivot Tables connected to cubes using VBA

Hi.

I recently embarked on the idea of building a pivot table connected to a cube. It seemed so simple !? Just build a MDX query, pass it to Excel, and voila!!! OOPS!!! Excel cannot take MDX queries. If you want to add rows, columns and measures, you have to do it it manually. And Then I started searching for the how to... Would you believe it... There are myriads of examples, and yet each one of them are only people who recorded a macro, then pasted the code. It helps to point you in the right direction, but I kept hitting stumpling blocks (as opposed to a stumbling block - a stumpling block is where you just don't know anymore... you are stumped).

So here is some code I developed. It builds the connection, clears the existing tables, and recreates a pivottable dynamically. The key though, is that I added 2 mods (AddFieldToTable and AddFilterToTable), that you just pass params to, and it will take care of the rest. Write once, use many times!!!

It may be easier to step through the code, than trying to explain it.

Please take a look at it,and comment if this helps...

Regards,

Zanoni...




Option Explicit
'An enum for the type of filter to be applied (one item or multiple items
Private Enum xlFilterType
SingleItemFilter = 0
MultipleItemFilter = 1
End Enum

'An enum to indicate if an item must be part of the list of visible items, or if it is an item to be removed
Private Enum xlFilterVisibility
Visible = 0
Hidden = 1
End Enum

'Collection to store the filter values
Private mColFilterFields As Collection

'An instance of an array for the values
Private MultipleFilterFields() As String


'==========================================================================================
'Main sub to build the Pivot Table
'==========================================================================================
'Date By Action
'==========================================================================================
'6 Jan 2010 Zanoni Labuschagne Created
'==========================================================================================
Public Sub GenerateResultSet()
On Error Resume Next
Dim strPartner, strYear, strScheme As String
Dim DestSheet As Excel.Worksheet
Dim DestRange As Excel.Range
Dim myPVT As Excel.PivotTable

'Set the sheet where the pivottable will be placed
Set DestSheet = ThisWorkbook.Worksheets("Sheet2")

'Set the range on the sheet where the pivottable will be placed
Set DestRange = DestSheet.Cells(10, 1)

Set mColFilterFields = New Collection

'Get the values for the filters
strPartner = DestSheet.Cells(1, 2).Value
strYear = DestSheet.Cells(2, 2).Value
strScheme = DestSheet.Cells(4, 2)

'Remove existing pivottables from the sheet
ClearExistingPivotTables DestSheet
DestRange.Select

'Build the connection to the cube
CreateConnection DestRange, "MyTable"

Set myPVT = DestSheet.PivotTables("MyTable")

'To make it look neater, hide the sheet while building
DestSheet.Visible = xlSheetHidden

'Hide the field list
ThisWorkbook.ShowPivotTableFieldList = False

'For performance, stop auto updating
myPVT.ManualUpdate = True

'Add the pivottable fields & filters
AddFieldToPivot myPVT, "Dimension Number1", "Attribute number 1", xlPageField
AddFilterToField myPVT, "Dimension Number1", "Attribute number 1", xlPageField, SingleItemFilter, "Some Value"

'Add another Dim Field
AddFieldToPivot myPVT, "Dimension Number 2", "Attribute number 2", xlPageField
AddFilterToField myPVT, "Dimension Number 2", "Attribute number 2", xlPageField, SingleItemFilter, "Some other Value"

'Add another Dim Field, and do a multiple filter
AddFieldToPivot myPVT, "Dimension Number 3", "Attribute number 3", xlRowField, True

'Hide e.g. the Unknown member
AddFilterToField myPVT, "Dimension Number 3", "Attribute number 3", xlRowField, MultipleItemFilter, "Unknown", Hidden

'Hide e.g. a blank member
AddFilterToField myPVT, "Dimension Number 3", "Attribute number 3", xlRowField, MultipleItemFilter, "", Hidden

'Hide e.g. SomeFunnyMember
AddFilterToField myPVT, "Dimension Number 3", "Attribute number 3", xlRowField, MultipleItemFilter, "SomeFunnyMember", Hidden

'Add some measures
AddFieldToPivot myPVT, "Measures", "[Measure 1]", xlDataField
AddFieldToPivot myPVT, "Measures", "[Measure 2]", xlDataField
AddFieldToPivot myPVT, "Measures", "[Measure 3]", xlDataField

'Update the table
myPVT.Update

'Show the sheet
DestSheet.Visible = xlSheetVisible

'Set the sheet focus
DestSheet.Select
End Sub


'==========================================================================================
'Adds a field to the PivotTable
'Parameters:
' PivotTableObject - A reference to the pivottable
' Dimension - Name of the dimension being added (For a measure, use "Measures"
' DimAttribute - Name of the attribute being added
' FieldType - Will the field be pleaced in the page area, Row, column, or value
' SuppressSubTotal - Must the subtotal be shown or hidden
'==========================================================================================
'Date By Action
'==========================================================================================
'6 Jan 2010 Zanoni Labuschagne Created
'==========================================================================================
Private Sub AddFieldToPivot(ByRef PivotTableObject As PivotTable, ByVal Dimension As String, ByVal DimAttribute As String, ByVal FieldType As XlPivotFieldOrientation, Optional ByVal SuppressSubTotal As Boolean)
On Error GoTo AddFieldError
Dim Hierarchy, FilterHierarchy As String

'Format and build the full hierarchy name
Dimension = "[" & Replace(Replace(Dimension, "]", ""), "[", "") & "]"
DimAttribute = "[" & Replace(Replace(DimAttribute, "]", ""), "[", "") & "]"
Hierarchy = Dimension & "." & DimAttribute
FilterHierarchy = Hierarchy & "." & DimAttribute


Select Case FieldType
'Add rows/Columns
Case xlColumnField, xlRowField
With PivotTableObject.CubeFields(Hierarchy)
.Orientation = FieldType
Select Case FieldType
Case xlColumnField
.Position = PivotTableObject.ColumnFields.Count + 1
Case xlRowField
.Position = PivotTableObject.RowFields.Count + 1
End Select
End With

If SuppressSubTotal Then
PivotTableObject.PivotFields(FilterHierarchy).Subtotals = Array(False, False, False, False, False, False, False, False, False, False, False, False)
End If

'Add Page Fields
Case xlPageField
With PivotTableObject.CubeFields(Hierarchy)
.Orientation = xlPageField
.Position = PivotTableObject.PageFields.Count + 1
End With

'Add measures
Case xlDataField
PivotTableObject.AddDataField PivotTableObject.CubeFields(Hierarchy)
End Select
Exit Sub

AddFieldError:
Select Case Err.Number
Case 9
MsgBox "Could not find the attribute " & DimAttribute & ".", vbCritical + vbOKOnly
Exit Sub
Case Else
MsgBox Err.Description, vbCritical + vbOKOnly
Resume Next
End Select
End Sub

'==========================================================================================
'Adds a filter to the PivotTable
'Parameters:
' PivotTableObject - A reference to the pivottable
' Dimension - Name of the dimension being added (For a measure, use "Measures"
' DimAttribute - Name of the attribute being added
' FieldType - Will the field be pleaced in the page area, Row, column, or value
' FilterType - is it one item, or multiple items
' Style - Must the filter remove an item from a full list, or add it to an emplty list
'==========================================================================================
'Date By Action
'==========================================================================================
'6 Jan 2010 Zanoni Labuschagne Created
'==========================================================================================

Private Sub AddFilterToField(ByRef PivotTableObject As PivotTable, ByVal Dimension As String, ByVal DimAttribute As String, ByVal FieldType As XlPivotFieldOrientation, ByVal FilterType As xlFilterType, ByVal FilterValue As String, Optional Style As xlFilterVisibility)
On Error GoTo AddFilterError
Dim Hierarchy, FilterHierarchy, FilterMember As String
Dim MultiFilterList() As String
'Format and build the full hierarchy name
Dimension = "[" & Replace(Replace(Dimension, "]", ""), "[", "") & "]"
DimAttribute = "[" & Replace(Replace(DimAttribute, "]", ""), "[", "") & "]"
Hierarchy = Dimension & "." & DimAttribute
FilterHierarchy = Hierarchy & "." & DimAttribute
FilterMember = Hierarchy & ".&[" & FilterValue & "]"

Select Case FilterType
Case SingleItemFilter
PivotTableObject.CubeFields(Hierarchy).EnableMultiplePageItems = False
PivotTableObject.PivotFields(FilterHierarchy).CurrentPageName = FilterMember

Case MultipleItemFilter
PivotTableObject.CubeFields(Hierarchy).EnableMultiplePageItems = True

MultipleFilterFields = GetFieldArrayFromCollection(Hierarchy)
ReDim Preserve MultipleFilterFields(UBound(MultipleFilterFields) + 1)
MultipleFilterFields(UBound(MultipleFilterFields)) = FilterMember
If Style = Visible Then
PivotTableObject.CubeFields(Hierarchy).IncludeNewItemsInFilter = False
PivotTableObject.CubeFields(Hierarchy).PivotFields(FilterHierarchy).VisibleItemsList = MultipleFilterFields
Else
PivotTableObject.CubeFields(Hierarchy).IncludeNewItemsInFilter = True
PivotTableObject.CubeFields(Hierarchy).PivotFields(FilterHierarchy).HiddenItemsList = MultipleFilterFields
End If
WriteFieldArrayToCollection Hierarchy, MultipleFilterFields()
End Select

Exit Sub

AddFilterError:
Select Case Err.Number
Case 9 ' Array not yet initialised
ReDim MultipleFilterFields(0)
Resume
Case Else
MsgBox "Could not add a filter to " & Dimension & " for value " & FilterMember & Chr(13) & Chr(13) & Err.Description, vbCritical + vbOKOnly
Exit Sub
End Select

End Sub

'==========================================================================================
'Creates the Excel Connection to the Cube
'Parameters:
' DestinationRange - The Excel Range where the cube must be placed
' TableName - The reference name you want to give the pivottable
'==========================================================================================
'Date By Action
'==========================================================================================
'6 Jan 2010 Zanoni Labuschagne Created
'==========================================================================================
Private Sub CreateConnection(ByRef DestinationRange As Range, ByVal TableName As String)
On Error Resume Next
'TODO: Add Variables instead of hardcoding
Dim ConnectionName, ConnectionString, ConnectionCommand As String

ConnectionName = "MyConnection"
ConnectionString = "OLEDB;Provider=MSOLAP.4;Integrated Security=SSPI;Persist Security Info=True;Data Source=YourServerName;Initial Catalog=YourASDatabase"
ConnectionCommand = "YourDefaultCube"

ThisWorkbook.Connections(ConnectionName).Delete
ThisWorkbook.Connections.Add ConnectionName, "Connection used to connect to HIP DW GP Cube", ConnectionString, ConnectionCommand, 1

ThisWorkbook.PivotCaches.Create(xlExternal, ThisWorkbook.Connections(ConnectionName), xlPivotTableVersion12).CreatePivotTable DestinationRange, TableName, xlPivotTableVersion12

End Sub
'==========================================================================================
'Clears all Pivot tables from the sheet
'Parameters:
' MySheet - The reference to the sheet to clear
'======================================e====================================================
'Date By Action
'==========================================================================================
'6 Jan 2010 Zanoni Labuschagne Created
'==========================================================================================
Private Sub ClearExistingPivotTables(ByRef mySheet As Worksheet)
Dim myPVT As PivotTable

For Each myPVT In mySheet.PivotTables
myPVT.PivotSelect "", xlDataAndLabel, True
Selection.ClearContents
Next myPVT
End Sub
'==========================================================================================
'Gets an array of filter fields from a collection
'Parameters:
' Hierarchy - The key to the item to be retrieved
'======================================e====================================================
'Date By Action
'==========================================================================================
'6 Jan 2010 Zanoni Labuschagne Created
'==========================================================================================
Private Function GetFieldArrayFromCollection(ByVal Hierarchy As String) As String()
On Error Resume Next
GetFieldArrayFromCollection = mColFilterFields(Hierarchy)
End Function


'==========================================================================================
'Writes an array of filter fields into a collection
'Parameters:
' Hierarchy - The key to the item to be retrieved
' FieldArray - The updated array to be written back into the collection
'======================================e====================================================
'Date By Action
'==========================================================================================
'6 Jan 2010 Zanoni Labuschagne Created
'==========================================================================================
Private Sub WriteFieldArrayToCollection(ByVal Hierarchy As String, FieldArray() As String)
On Error Resume Next
mColFilterFields.Remove Hierarchy
mColFilterFields.Add FieldArray(), Hierarchy
End Sub

Dynamics Integration Manager Niggles

Being a newbie to GP IM (yet another ETL tool), I recently battled to get journals through to GP, and here is what I learned:

• If you ever have an issue where GP doesn't return your columns, check your query... I had one calling a stored proc, and in the proc, had multiple statements... when running the proc in SSMS, it works, but in GP IM it doesn't... At the top of the proc include:
  "Set Nocount On"
  
  It turns out, it only runs up to the result it gets, then stops. This statement means don't return any row counts.
  


• Make sure that, under Destination Mapping -> Entries-> Options, you have use "Use Source Recordset" for Recordset. If you don't, the batches will be created but won't have any transactions in them.
  


• GP's ODBC connection can't have ANSI NULL translations. I had a "where" clause in the stored proc source stripping rows with no account number, but the rows were still being returned!?. I had to find another way to remove these rows (I joined to another source table to filter empty rows out)
  

Excel 2007 Hangs when opening a .XLSM workbook

I recently had a massive amount of scripting in a workbook. I saved it, but when I tried reopening it, it froze. I almost cried.

First, try repoening it while pressing Shift (this disables the macros)

If no success

To fix it:
* Click the Start Menu then select Run. Type "regedit" (without the quotation marks) into the Run dialog box and click OK.
* Select My Computer at the top of the Registry tree in the left pane of the Registry Editor.
* Then backup the registry by selecting File - Export from the Registry Editor menu, entering a name for the registry backup file and clicking OK.
* Navigate the the following registry key: HKEY_CURRENT_USER\Software\Microsoft\Office\12.0\Excel\Security
* With the "Security" key highlighted, select Edit - New - DWORD Value from the Registry Editor menu.
* Type "ExcelBypassEncryptedMacroScan" (without the quotation marks) for the new value name and hit enter
* Then double-click on the newly added value and change the DWORD Value from 0 to 1 and then click OK.
* Exit the Registry Editor and the exit and restart Excel 2007 for the change to take affect.