Interesting programming ideas, solutions, and logic that I have used to solve problems or have come across throughout my career.
About Me
- William Andrus
- Northglenn, Colorado, United States
- I'm primarily a BI Developer on the Microsoft stack. I do sometimes touch upon other Microsoft stacks ( web development, application development, and sql server development).
Tuesday, October 21, 2014
Optimizing SSRS Rendering and Performance
Tuesday, August 14, 2012
First StreamInsight attempt using Performance Counter
3 important downloads to get started, with StreamInsight:
Download Examples: http://streaminsight.codeplex.com/releases/view/90143
First I created an class to represent the data I wanted to capture in my stream. Not all data types can be used with StreamInsight (ie: enum). Here is a list of supported data types: http://msdn.microsoft.com/en-us/library/ee378905.aspx
- // Input events of CounterSample of supported data types
- public class StreamableCounterSample
- {
- public float CpuUtilization { get; set; }
- public long TimeStamp { get; set; }
- }
The main portion of the program.
Here starting on line 86, I create my StreamInsight server embedded in memory. The server name is the instance name that you give after installing StreamInsight. With StreamInsight 2.1, they added IQStreamable which allows you to query the stream. Lines 104 & 105 is when the stream will be queried for the results.
- private float CalculatePerformance(BackgroundWorker worker, DoWorkEventArgs e)
- {
- CreateCounters();
- //embedded (in-memory)
- using (Server server = Server.Create("StreamInsight21"))
- {
- Microsoft.ComplexEventProcessing.Application application = server.CreateApplication("app");
- //A query for reading events from a stream
- IQStreamable<StreamableCounterSample> inputStream = null;
- inputStream = CreateStream(application);
- while (true)
- {
- if (worker.CancellationPending)
- {
- e.Cancel = true;
- break;
- }
- else
- {
- perf = inputStream.ToObservable().ToEnumerable().Last().CpuUtilization;
- timestamp = DateTime.FromFileTime(inputStream.ToObservable().ToEnumerable().Last().TimeStamp);
- worker.ReportProgress((int)perf);
- }
- }
- return inputStream.ToObservable().ToEnumerable().Last().CpuUtilization;
- }
- }
When creating the stream in line 93, I'm setting up my observation. In this case CollectSamples is my source which is then converted to a temporal stream via ToPointStreamable, which inserts a single event instance with a datetime.
- static IQStreamable<StreamableCounterSample> CreateStream(Microsoft.ComplexEventProcessing.Application application)
- {
- // Live data uses IQbservable<>
- return
- application.DefineObservable(() => CollectSamples()).ToPointStreamable(
- r => PointEvent<StreamableCounterSample>.CreateInsert(DateTime.Now, r),
- AdvanceTimeSettings.StrictlyIncreasingStartTime);
- }
The collect samples, is where my data will be pulled. In this case I'm pulling my data from the performance counter, but returning an observable interval representing this temporal event that occurs every 1 second.
- private static IObservable<StreamableCounterSample> CollectSamples()
- {
- List<StreamableCounterSample> data = new List<StreamableCounterSample>();
- data.Add(new StreamableCounterSample
- {
- CpuUtilization = perfCounter.NextValue(),
- TimeStamp = DateTime.Now.ToFileTime()
- });
- return ToObservableInterval(data, TimeSpan.FromMilliseconds(1000), Scheduler.ThreadPool);
- }
- private static IObservable
ToObservableInterval (IEnumerable source, TimeSpan period, IScheduler scheduler) - {
- return Observable.Using(
- () => source.GetEnumerator(),
- it => Observable.Generate(
- default(object),
- _ => it.MoveNext(),
- _ => _,
- _ =>
- {
- //Console.WriteLine("Input {0}", it.Current);
- return it.Current;
- },
- _ => period, scheduler));
- }
In the end, the final product is a simple winform showing the current CPU Utilization and the last time it ran.
The program can be downloaded from:
Thursday, March 08, 2012
View overall I/O statistics for all databases in your server
"drafting a server consolidation strategy, or pointing to eventual need to check perf counters if I/O is waiting more than expected"- From "Blog do Ezequiel" http://blogs.msdn.com/b/blogdoezequiel/archive/2012/03/08/the-sql-swiss-army-knife-3-view-i-o-per-file-updated.aspx
--
-- Checks the cumulative IO per database file and related information
--
SELECT f.database_id, DB_NAME(f.database_id) AS database_name, f.name AS logical_file_name, f.[file_id], f.type_desc,
CAST (CASE
-- Handle UNC paths (e.g. '\\fileserver\readonlydbs\dept_dw.ndf')
WHEN LEFT (LTRIM (f.physical_name), 2) = '\\'
THEN LEFT (LTRIM (f.physical_name),CHARINDEX('\',LTRIM(f.physical_name),CHARINDEX('\',LTRIM(f.physical_name), 3) + 1) - 1)
-- Handle local paths (e.g. 'C:\Program Files\...\master.mdf')
WHEN CHARINDEX('\', LTRIM(f.physical_name), 3) > 0
THEN UPPER(LEFT(LTRIM(f.physical_name), CHARINDEX ('\', LTRIM(f.physical_name), 3) - 1))
ELSE f.physical_name
END AS NVARCHAR(255)) AS logical_disk,
fs.size_on_disk_bytes/1024/1024 AS size_on_disk_Mbytes,
fs.num_of_reads, fs.num_of_writes,
fs.num_of_bytes_read/1024/1024 AS num_of_Mbytes_read,
fs.num_of_bytes_written/1024/1024 AS num_of_Mbytes_written,
fs.io_stall/1000/60 AS io_stall_min,
fs.io_stall_read_ms/1000/60 AS io_stall_read_min,
fs.io_stall_write_ms/1000/60 AS io_stall_write_min,
((fs.io_stall_read_ms/1000/60)*100)/(CASE WHEN fs.io_stall/1000/60 = 0 THEN 1 ELSE fs.io_stall/1000/60 END) AS io_stall_read_pct,
((fs.io_stall_write_ms/1000/60)*100)/(CASE WHEN fs.io_stall/1000/60 = 0 THEN 1 ELSE fs.io_stall/1000/60 END) AS io_stall_write_pct,
ABS((sample_ms/1000)/60/60) AS 'sample_HH',
((fs.io_stall/1000/60)*100)/(ABS((sample_ms/1000)/60))AS 'io_stall_pct_of_overall_sample'
FROM sys.dm_io_virtual_file_stats (default, default) AS fs
INNER JOIN sys.master_files AS f ON fs.database_id = f.database_id AND fs.[file_id] = f.[file_id]
ORDER BY 18 DESC
Wednesday, October 17, 2007
Concat strings in Javascript performance.
So basically they are fixing the performance speed it takes to concat strings in javascript for the next release of IE. The improvement is very significant.
Monday, October 15, 2007
Optimizing SQL Server CPU Performance
Some interesting FYI facts from the article:
- In performance value: high-end dual-core processor > RAM > fibre optics > disk drive
- A data page in SQL Server is 8KB.
- An extent in SQL Server is made up of eight 8KB pages, making it equivalent to 64KB.
- Pulling a data page that is already cached from the buffer pool, at peak performance, should take under half a millisecond; retrieving a single extent from disk should take between 2 and 4 milliseconds in an optimal environment.
Check CPU utilization with PerfMon by monitoring: % Processor Time (<80%),>
Some optimizations are:
Query plan reuse
Reducing compiles and recompiles
Sort operations
Improper joins
Missing indexes
Table/index scans
Function usage in SELECT and WHERE clauses
Multithreaded operation
Resources for information on query plan reuse:
Batch Compilation, Recompilation, and Plan Caching Issues in SQL Server 2005 (microsoft.com/technet/prodtechnol/sql/2005/recomp.mspx)
Optimizing SQL Server Stored Procedures to Avoid Recompiles (sql-server-performance.com/rd_optimizing_sp_recompiles.asp)
Query Recompilation in SQL Server 2000 (msdn2.microsoft.com/aa902682.aspx)
