Asynchronous MultiThreaded SSH engine for Web (Net Core 6, Linux) - Part 1,2 (Database and SSH client). StreamReader/ReadToEndAsync, Extension/Inherits/Overloads, BeginExecute/EndExecute/IAsyncResult/IProgress(Of T)/Async/Await/CancellationToken/Task.Run/Thread.Yield.
I like write various Asynchronous MultiThreaded engine, look for example this my posts.
- 2018 Multithreading Parsers with Parallel, CsQuery, Newtonsoft.Json, OfficeOpenXml and IAsyncResult/AsyncCallback
- 2019 ASP.NET Core Backend with MailKit/IMAP (Async/Await, Monitor, ConcurrentQueue, CancellationTokenSource)
And whole my life I created various implementation of PUSH notification from server to client, look for example this my post.
- 2007 Вариации на тему Notification-сервера
- 2005 TCP/IP Notification Server для SQL2000 на COM-обьекте
Of course, if you use MsSQL server, the best way is using this future Use SqlDependency/SqlServiceBroker to send notification to SignalR hub, but similar future is absent in other SQL Server, this is, of course, is hard proof why EF Code First is absolutely idiotic idea, because there nothing similar function in different SQL Server (except store data to tables). At common EF Code First has so many other restrictions, for example this engine even don't understand what is View, impossible to work with existing database (EF DB First theoretically allow working, but in practice there are nothing EF providers for other DB, except DB from Microsoft). Therefore, if you want to use similar futures like SqlDependency/SqlServiceBrokerSqlDependency/SqlServiceBroker for other DB, you need to realize own engine. But disadvantage of EF is another story - Класифікація засобів роботи з даними.Класифікація засобів роботи з даними, in next parts I will describe my way to access Data more details.
This Engine I created as evolution and developing of two projects I described so far in my blog:
- 2021 Linux shell parser (Renci.SshNet.SshClient, MySqlConnector.NET, RSA encription, Console password, XDocument/XPath, Tuple, Regex, Buffer.BlockCopy, Stream, Base64String, UTF8.GetBytes, ToString("X2"))
- 2022 BackendAPI (Net Core 5) project template with Custom Attribute, Service and Controller Example, MySQL database with masked password in config, SwaggerUI, EF Core and extension to EF CoreBackendAPI (Net Core 5) project template with Custom Attribute, Service and Controller Example, MySQL database with masked password in config, SwaggerUI, EF Core and extension to EF Core
Engine I described in this page allow to order any SSH command to Queue to each VM and Servers, than perform each SSH command one-by-one for each BareMetal Server or each project VM, some command can working at many hours, for example DD like this, and than client receive notification with result by SignalR. In the screen below you can see how this engine ordered SSH command to a Queue for a couple VM and Servers, than perform it one-by-one, and send result to backend.
1. Database.
So, The first part of this Engine is Database MySQL Kvm hosting management Db with tracking changing by triggers. This Database is huge and not all tables is need to this Engine, this Engine use only one table
Function of this Engine also related with table User (this is reference to ToUser), table VM (this is reference ToVm of Virtual Machines of project) and table KvmHost (this is reference ToServer of Bare Metal Server of project). Password to access for Server or VM will be produced from fiels SshDecryptPass. I hope reader understanding goal of other fields.
For Access with MySQL I use this way. I will write more details about masking real password to Db later. This is one of this keys to protect project from rogue intrusion.
My project use this providers to access MySQL (MariaDB).
2. SSH client.
Next component of this Engine is Asynchronous SSH client. As access to bash I use https://github.com/sshnet/SSH.NET/, but this is only one line of code in this library. I wrap SSH engine to separate library and uploaded this component to Github https://github.com/Alex-1557/AsyncSshEngine.
This component composed from six classes/modules.
This library refs to two POCO-classes with describe information about BareMetal Servers and Virtual Machines. And final classes separated to two namespaces - related to Vm and related to Servers.
Also this library used two Net core service ApplicationDbContext and AesCryptor, I will describe this services in the next pages.
So, now we look to this library. ServerBash is my base class, when it creating it reading information from DB about server it describe. Similar class is exists in VM namespace with little VM specific.
In derived class VmBashAsync I redefine main method Bash from base class ServerBash with Asynchronous pattern. In this class I use my own extension ExecuteAsync, system class Progress, and class to collect output of SSH.NET SshOutputLine.
1: Imports BackendAPI.Model
2: Imports BackendAPI.Services
3:
4: Namespace Kvm
5: Public Class ServerBashAsync
6: Inherits ServerBash
7:
8: Public SshOutput As List(Of String)
9: Public SshErrMsg As List(Of String)
10: Public Sub New(_DB As ApplicationDbContext, _Aes As AesCryptor, ServerI As Integer, ServerDecryptPass As String)
11: MyBase.New(_DB, _Aes, ServerI, ServerDecryptPass)
12: SshErrMsg = New List(Of String)
13: SshOutput = New List(Of String)
14: End Sub
15:
16: Public Overloads Async Function Bash(BashCmd As String) As Task(Of String)
17: Dim CTX = New Threading.CancellationTokenSource()
18: If SshClient.IsConnected Then
19: Try
20: Dim Cmd1 = SshClient.CreateCommand(BashCmd)
21: Await Task.Run(Function() Cmd1.ExecuteAsync(New Progress(Of SshOutputLine), CTX.Token, SshOutput, SshErrMsg, KvmHosts.MainServerIP))
22: 'Await Cmd1.ExecuteAsync(New Progress(Of SshOutputLine), CTX)
23: Catch ex As Exception
24: Return ex.Message
25: End Try
26: Else
27: Await Task.Run(Function() "not connected")
28: End If
29: End Function
30:
31: End Class
32: End Namespace
SshOutputLine is a simple collector of output with divide Error and Normal output to two class (debugging is commented, it can show to application console each portion of information receiving from BASH interpreter).
1: Public Class SshOutputLine
2: Public Shared Property LineCount As Integer
3:
4: Public Sub New(ByVal Line As String, ByVal IsErrorLine As Boolean, SshOutput As List(Of String), SshErrMsg As List(Of String), IpAddr As String)
5: 'Debug.WriteLine($"Line Ready ({Now}): {Line}")
6: LineCount += 1
7: Debug.WriteLine($"{IpAddr} Line Ready ({Now}): {LineCount}")
8:
9: If IsErrorLine Then
10: SshErrMsg.Add(Line)
11: Else
12: SshOutput.Add(Line)
13: End If
14: End Sub
15: End Class
If we uncomment debugging we will see LineReady message - order where each line will from Bash interpreter will be ready.
And last component is module with Async extension SshCommandExtensions.
1: Imports System.IO
2: Imports System.Runtime.CompilerServices
3: Imports System.Threading
4: Imports Renci.SshNet
5:
6: Module SshCommandExtensions
7: <Extension()>
8: Async Function ExecuteAsync(ByVal SshCommand As SshCommand, ByVal OutputLine As IProgress(Of SshOutputLine), ByVal CTX As CancellationToken, SshOutput As List(Of String), SshErrMsg As List(Of String), IpAddr As String) As Task
9: Dim AsyncResult As IAsyncResult = SshCommand.BeginExecute()
10: Dim StdoutSR = New StreamReader(SshCommand.OutputStream)
11: Dim StderrSR = New StreamReader(SshCommand.ExtendedOutputStream)
12: While Not AsyncResult.IsCompleted
13: 'Debug.Print($"#1 {AsyncResult.IsCompleted} {Now} {String.Join(".", SshOutput)}")
14: ' Debug.Print($"#1 {IpAddr} {AsyncResult.IsCompleted} {Now} ")
15: Await Progress(SshCommand, StdoutSR, StderrSR, OutputLine, CTX, SshOutput, SshErrMsg, IpAddr)
16: Thread.Yield()
17: 'Debug.Print($"#2 {AsyncResult.IsCompleted} {Now} {String.Join(".", SshOutput)}")
18: 'Debug.Print($"#2 {IpAddr} {AsyncResult.IsCompleted} {Now} ")
19: End While
20: 'Debug.Print($"#3 {AsyncResult.IsCompleted} {Now} {String.Join(".", SshOutput)}")
21: 'Debug.Print($"#3 {IpAddr} {AsyncResult.IsCompleted} {Now} ")
22: SshCommand.EndExecute(AsyncResult)
23: Await Progress(SshCommand, StdoutSR, StderrSR, OutputLine, CTX, SshOutput, SshErrMsg, IpAddr)
24: 'Debug.Print($"#4 {AsyncResult.IsCompleted} {Now} {String.Join(".", SshOutput)}")
25: 'Debug.Print($"#4 {IpAddr} {AsyncResult.IsCompleted} {Now} ")
26: End Function
27:
28: Private Async Function Progress(ByVal SshCommand As SshCommand, ByVal StdoutSR As TextReader, ByVal StderrSR As TextReader, ByVal OutputLine As IProgress(Of SshOutputLine), ByVal CTX As CancellationToken, SshOutput As List(Of String), SshErrMsg As List(Of String), IpAddr As String) As Task
29: If CTX.IsCancellationRequested Then SshCommand.CancelAsync()
30: CTX.ThrowIfCancellationRequested()
31: Await OutProgress(StdoutSR, OutputLine, SshOutput, SshErrMsg, IpAddr)
32: Await ErrProgress(StderrSR, OutputLine, SshOutput, SshErrMsg, IpAddr)
33: End Function
34:
35: Private Async Function OutProgress(ByVal StdoutSR As TextReader, ByVal StdoutProgress As IProgress(Of SshOutputLine), SshOutput As List(Of String), SshErrMsg As List(Of String), IpAddr As String) As Task
36: Dim StdoutLine = Await StdoutSR.ReadToEndAsync()
37: If Not String.IsNullOrEmpty(StdoutLine) Then StdoutProgress.Report(New SshOutputLine(Line:=StdoutLine, IsErrorLine:=False, SshOutput, SshErrMsg, IpAddr))
38: End Function
39:
40: Private Async Function ErrProgress(ByVal StderrSR As TextReader, ByVal stderrProgress As IProgress(Of SshOutputLine), SshOutput As List(Of String), SshErrMsg As List(Of String), IpAddr As String) As Task
41: Dim StderrLine = Await StderrSR.ReadToEndAsync()
42: If Not String.IsNullOrEmpty(StderrLine) Then stderrProgress.Report(New SshOutputLine(Line:=StderrLine, IsErrorLine:=True, SshOutput, SshErrMsg, IpAddr))
43: End Function
44: End Module
Debugging is commented.
That's it, description of this engine will be continue in next pages.
)
| <SITEMAP> <MVC> <ASP> <NET> <DATA> <KIOSK> <FLEX> <SQL> <NOTES> <LINUX> <MONO> <FREEWARE> <DOCS> <ENG> <CHAT ME> <ABOUT ME> < THANKS ME> |