IT虾米网

网络编程基础详解

leader 2018年07月09日 编程语言 272 0
一、socket

socket简介:

Socket又称"套接字",应用程序通常通过"套接字"向网络发出请求或者应答网络请求。

socket本质上就是在2台网络互通的电脑之间,架设一个通道,两台电脑通过这个通道来实现数据的互相传递。 我们知道网络 通信 都 是基于 ip+port 方能定位到目标的具体机器上的具体服务,操作系统有0-65535个端口,每个端口都可以独立对外提供服务,如果 把一个公司比做一台电脑 ,那公司的总机号码就相当于ip地址, 每个员工的分机号就相当于端口, 你想找公司某个人,必须 先打电话到总机,然后再转分机 。

建立一个socket必须至少有2端, 一个服务端,一个客户端, 服务端被动等待并接收请求,客户端主动发起请求, 连接建立之后,双方可以互发数据。 

python提供两个socket模块:

1.socket:提供了标准的 BSD Sockets API。

2. socketserver:它提供了服务器中心类,可以简化网络服务器的开发。

socket通讯过程:

 

socket模块使用:

socket对象:

sock = socket.socket(socket.AF_INET,socket.SOCK_STREAM,0)

参数一:地址簇

参数 描述
socket.AF_INET IPv4(默认)
socket.AF_INET6 IPv6
socket.AF_UNIX 只能够用于单一的Unix系统进程间通信

参数二:类型

参数 描述
socket.SOCK_STREAM 流式socket , for TCP (默认)
socket.SOCK_DGRAM 数据报式socket , for UDP
socket.SOCK_RAW 原始套接字,普通的套接字无法处理ICMP、IGMP等网络报文,而SOCK_RAW可以;其次,SOCK_RAW也可以处理特殊的IPv4报文;此外,利用原始套接字,可以通过IP_HDRINCL套接字选项由用户构造IP头。
socket.SOCK_RDM 是一种可靠的UDP形式,即保证交付数据报但不保证顺序。SOCK_RAM用来提供对原始协议的低级访问,在需要执行某些特殊操作时使用,如发送ICMP报文。SOCK_RAM通常仅限于高级用户或管理员运行的程序使用。
socket.SOCK_SEQPACKET 可靠的连续数据包服务

参数三:协议

参数 描述
0 (默认)与特定的地址家族相关的协议,如果是 0 ,则系统就会根据地址格式和套接类别,自动选择一个合适的协议

socket中基本方法

方法 描述
s.bind(address) 将套接字绑定到地址。address地址的格式取决于地址族。在AF_INET下,以元组(host,port)的形式表示地址。
sk.listen(backlog) 开始监听传入连接。backlog指定在拒绝连接之前,可以挂起的最大连接数量。
sk.setblocking(bool) 是否阻塞(默认True),如果设置False,那么accept和recv时一旦无数据,则报错。
sk.accept() 接受连接并返回(conn,address),其中conn是新的套接字对象,可以用来接收和发送数据。address是连接客户端的地址。
sk.connect(address) 连接到address处的套接字。一般,address的格式为元组(hostname,port),如果连接出错,返回socket.error错误。
sk.connect_ex(address) 同上,只不过会有返回值,连接成功时返回 0 ,连接失败时候返回编码,例如:10061
sk.close() 关闭套接字连接
sk.recv(bufsize[,flag]) 接受套接字的数据。数据以字符串形式返回,bufsize指定最多可以接收的数量。flag提供有关消息的其他信息,通常可以忽略。
sk.recvfrom(bufsize[.flag]) 与recv()类似,但返回值是(data,address)。其中data是包含接收数据的字符串,address是发送数据的套接字地址。
sk.send(string[,flag]) 将string中的数据发送到连接的套接字。返回值是要发送的字节数量,该数量可能小于string的字节大小。即:可能未将指定内容全部发送。
sk.sendall(string[,flag]) 将string中的数据发送到连接的套接字,但在返回之前会尝试发送所有数据。成功返回None,失败则抛出异常。内部通过递归调用send,将所有内容发送出去。
sk.sendto(string[,flag],address) 将数据发送到套接字,address是形式为(ipaddr,port)的元组,指定远程地址。返回值是发送的字节数。该函数主要用于UDP协议。
sk.settimeout(timeout) 设置套接字操作的超时期,timeout是一个浮点数,单位是秒。值为None表示没有超时期。
sk.getpeername() 返回连接套接字的远程地址。返回值通常是元组(ipaddr,port)。
sk.getsockname() 返回套接字自己的地址。通常是一个元组(ipaddr,port)
sk.fileno() 套接字的文件描述符

简单socket通讯demo:

server端:

#!/usr/bin/env python3 
#_*_ coding:utf-8 _*_ 
#Author:wd 
import socket 
server=socket.socket()#生成socket实例 
server.bind(('127.0.0.1',6666))#绑定地址 
server.listen(5)#设置支持连接个数 
while True:#一次层循环用户等待客户端连接 
    print('等待客户端连接.....') 
    con,addr=server.accept() 
    while True:#第二层循环用于和客户端交互 
        data=con.recv(1024) 
        if data: 
            con.send(data) 
        else: 
            con.close() 
            print('客户端已经断开连接.....') 
            break

client端:

#!/usr/bin/env python3 
#_*_ coding:utf-8 _*_ 
#Author:wd 
import json 
import socket 
client=socket.socket()#生成socket实例 
client.connect(('127.0.0.1',6666))#连接服务器 
while True:#while循环用于和客户端一直交互 
    data=input('>>>').strip() 
    if len(data)==0: 
        continue 
    else: 
        data=json.dumps(data) 
        client.send(data.encode())#python3数据传输为bytes类型需要encode 
        recv_data=client.recv(1024) 
        if len(recv_data)==0: 
            print('没有数据!') 
            continue 
        else:print(data)

2.socketserver 

为了简化socket的开发,socketserver提供了多种socket服务器模块,关系如下:

+------------+ 
| BaseServer | 
+------------+ 
      | v +-----------+ +------------------+ | TCPServer |------->| UnixStreamServer | +-----------+ +------------------+ | v +-----------+ +--------------------+ | UDPServer |------->| UnixDatagramServer | +-----------+ +--------------------+


SocketServer内部使用IO多路复用、以及 “多线程” 和 “多进程”(后续篇章会提到),从而实现并发处理多个客户端请求的Socket服务端。即:每个客户端请求连接到服务器时,Socket服务端都会在服务器是创建一个“线程”或者“进程” 专门负责处理当前客户端的所有请求。

1.ThreadingTCPServer介绍:

ThreadingTCPServer是socketserver中最常用的模块,它是多线程,并且支持多并发。

使用ThreadingTCPServer:

  • 创建一个继承自 SocketServer.BaseRequestHandler 的类
  • 类中必须定义一个名称为 handle 的方法,用于处理客户端的交互。
  • 启动ThreadingTCPServer

一个简单的TreadingTCPServer例子如下:

import socketserver 
class Ftp_server(socketserver.BaseRequestHandler): 
    '''FTP server 主程序类定义''' 
    def handle(self):#重写父类handle方法,处理客户端请求 
        print("等待客户端连接....") 
        while True:#进入循环和客户端交互 
            try: 
                self.data=self.request.recv(2048) 
                if self.data: 
                    data = json.loads(self.data.decode()) 
                    print(data) 
                    if hasattr(self,data.get('action')): 
                        fun=getattr(self,data.get('action')) 
                        fun(data) 
            else:break 
server=socketserver.ThreadingTCPServer((settings.HOST,settings.PORT),Ftp_server) 
server.serve_forever()#启动监听

 

TreadingTCPServer源码解析:

模块关系:

 

内部调用流程为:

  • 启动服务端程序
  • 执行 TCPServer.__init__ 方法,创建服务端Socket对象并绑定 IP 和 端口
  • 执行 BaseServer.__init__ 方法,将自定义的继承自SocketServer.BaseRequestHandler 的类 MyRequestHandle赋值给self.RequestHandlerClass
  • 执行 BaseServer.server_forever 方法,While 循环一直监听是否有客户端请求到达 ...
  • 当客户端连接到达服务器
  • 执行 ThreadingMixIn.process_request 方法,创建一个 “线程” 用来处理请求
  • 执行 ThreadingMixIn.process_request_thread 方法
  • 执行 BaseServer.finish_request 方法,执行 self.RequestHandlerClass()  即:执行 自定义 MyRequestHandler 的构造方法(自动调用基类BaseRequestHandler的构造方法,在该构造方法中又会调用 MyRequestHandler的handle方法)

相关模块源码

class BaseServer: 
 
    """Base class for server classes. 
 
    Methods for the caller: 
 
    - __init__(server_address, RequestHandlerClass) 
    - serve_forever(poll_interval=0.5) 
    - shutdown() 
    - handle_request()  # if you do not use serve_forever() 
    - fileno() -> int   # for select() 
 
    Methods that may be overridden: 
 
    - server_bind() 
    - server_activate() 
    - get_request() -> request, client_address 
    - handle_timeout() 
    - verify_request(request, client_address) 
    - server_close() 
    - process_request(request, client_address) 
    - shutdown_request(request) 
    - close_request(request) 
    - handle_error() 
 
    Methods for derived classes: 
 
    - finish_request(request, client_address) 
 
    Class variables that may be overridden by derived classes or 
    instances: 
 
    - timeout 
    - address_family 
    - socket_type 
    - allow_reuse_address 
 
    Instance variables: 
 
    - RequestHandlerClass 
    - socket 
 
    """ 
 
    timeout = None 
 
    def __init__(self, server_address, RequestHandlerClass): 
        """Constructor.  May be extended, do not override.""" 
        self.server_address = server_address 
        self.RequestHandlerClass = RequestHandlerClass 
        self.__is_shut_down = threading.Event() 
        self.__shutdown_request = False 
 
    def server_activate(self): 
        """Called by constructor to activate the server. 
 
        May be overridden. 
 
        """ 
        pass 
 
    def serve_forever(self, poll_interval=0.5): 
        """Handle one request at a time until shutdown. 
 
        Polls for shutdown every poll_interval seconds. Ignores 
        self.timeout. If you need to do periodic tasks, do them in 
        another thread. 
        """ 
        self.__is_shut_down.clear() 
        try: 
            while not self.__shutdown_request: 
                # XXX: Consider using another file descriptor or 
                # connecting to the socket to wake this up instead of 
                # polling. Polling reduces our responsiveness to a 
                # shutdown request and wastes cpu at all other times. 
                r, w, e = _eintr_retry(select.select, [self], [], [], 
                                       poll_interval) 
                if self in r: 
                    self._handle_request_noblock() 
        finally: 
            self.__shutdown_request = False 
            self.__is_shut_down.set() 
 
    def shutdown(self): 
        """Stops the serve_forever loop. 
 
        Blocks until the loop has finished. This must be called while 
        serve_forever() is running in another thread, or it will 
        deadlock. 
        """ 
        self.__shutdown_request = True 
        self.__is_shut_down.wait() 
 
    # The distinction between handling, getting, processing and 
    # finishing a request is fairly arbitrary.  Remember: 
    # 
    # - handle_request() is the top-level call.  It calls 
    #   select, get_request(), verify_request() and process_request() 
    # - get_request() is different for stream or datagram sockets 
    # - process_request() is the place that may fork a new process 
    #   or create a new thread to finish the request 
    # - finish_request() instantiates the request handler class; 
    #   this constructor will handle the request all by itself 
 
    def handle_request(self): 
        """Handle one request, possibly blocking. 
 
        Respects self.timeout. 
        """ 
        # Support people who used socket.settimeout() to escape 
        # handle_request before self.timeout was available. 
        timeout = self.socket.gettimeout() 
        if timeout is None: 
            timeout = self.timeout 
        elif self.timeout is not None: 
            timeout = min(timeout, self.timeout) 
        fd_sets = _eintr_retry(select.select, [self], [], [], timeout) 
        if not fd_sets[0]: 
            self.handle_timeout() 
            return 
        self._handle_request_noblock() 
 
    def _handle_request_noblock(self): 
        """Handle one request, without blocking. 
 
        I assume that select.select has returned that the socket is 
        readable before this function was called, so there should be 
        no risk of blocking in get_request(). 
        """ 
        try: 
            request, client_address = self.get_request() 
        except socket.error: 
            return 
        if self.verify_request(request, client_address): 
            try: 
                self.process_request(request, client_address) 
            except: 
                self.handle_error(request, client_address) 
                self.shutdown_request(request) 
 
    def handle_timeout(self): 
        """Called if no new request arrives within self.timeout. 
 
        Overridden by ForkingMixIn. 
        """ 
        pass 
 
    def verify_request(self, request, client_address): 
        """Verify the request.  May be overridden. 
 
        Return True if we should proceed with this request. 
 
        """ 
        return True 
 
    def process_request(self, request, client_address): 
        """Call finish_request. 
 
        Overridden by ForkingMixIn and ThreadingMixIn. 
 
        """ 
        self.finish_request(request, client_address) 
        self.shutdown_request(request) 
 
    def server_close(self): 
        """Called to clean-up the server. 
 
        May be overridden. 
 
        """ 
        pass 
 
    def finish_request(self, request, client_address): 
        """Finish one request by instantiating RequestHandlerClass.""" 
        self.RequestHandlerClass(request, client_address, self) 
 
    def shutdown_request(self, request): 
        """Called to shutdown and close an individual request.""" 
        self.close_request(request) 
 
    def close_request(self, request): 
        """Called to clean up an individual request.""" 
        pass 
 
    def handle_error(self, request, client_address): 
        """Handle an error gracefully.  May be overridden. 
 
        The default is to print a traceback and continue. 
 
        """ 
        print '-'*40 
        print 'Exception happened during processing of request from', 
        print client_address 
        import traceback 
        traceback.print_exc() # XXX But this goes to stderr! 
        print '-'*40
BaseServer
class TCPServer(BaseServer): 
 
    """Base class for various socket-based server classes. 
 
    Defaults to synchronous IP stream (i.e., TCP). 
 
    Methods for the caller: 
 
    - __init__(server_address, RequestHandlerClass, bind_and_activate=True) 
    - serve_forever(poll_interval=0.5) 
    - shutdown() 
    - handle_request()  # if you don't use serve_forever() 
    - fileno() -> int   # for select() 
 
    Methods that may be overridden: 
 
    - server_bind() 
    - server_activate() 
    - get_request() -> request, client_address 
    - handle_timeout() 
    - verify_request(request, client_address) 
    - process_request(request, client_address) 
    - shutdown_request(request) 
    - close_request(request) 
    - handle_error() 
 
    Methods for derived classes: 
 
    - finish_request(request, client_address) 
 
    Class variables that may be overridden by derived classes or 
    instances: 
 
    - timeout 
    - address_family 
    - socket_type 
    - request_queue_size (only for stream sockets) 
    - allow_reuse_address 
 
    Instance variables: 
 
    - server_address 
    - RequestHandlerClass 
    - socket 
 
    """ 
 
    address_family = socket.AF_INET 
 
    socket_type = socket.SOCK_STREAM 
 
    request_queue_size = 5 
 
    allow_reuse_address = False 
 
    def __init__(self, server_address, RequestHandlerClass, bind_and_activate=True): 
        """Constructor.  May be extended, do not override.""" 
        BaseServer.__init__(self, server_address, RequestHandlerClass) 
        self.socket = socket.socket(self.address_family, 
                                    self.socket_type) 
        if bind_and_activate: 
            try: 
                self.server_bind() 
                self.server_activate() 
            except: 
                self.server_close() 
                raise 
 
    def server_bind(self): 
        """Called by constructor to bind the socket. 
 
        May be overridden. 
 
        """ 
        if self.allow_reuse_address: 
            self.socket.setsockopt(socket.SOL_SOCKET, socket.SO_REUSEADDR, 1) 
        self.socket.bind(self.server_address) 
        self.server_address = self.socket.getsockname() 
 
    def server_activate(self): 
        """Called by constructor to activate the server. 
 
        May be overridden. 
 
        """ 
        self.socket.listen(self.request_queue_size) 
 
    def server_close(self): 
        """Called to clean-up the server. 
 
        May be overridden. 
 
        """ 
        self.socket.close() 
 
    def fileno(self): 
        """Return socket file number. 
 
        Interface required by select(). 
 
        """ 
        return self.socket.fileno() 
 
    def get_request(self): 
        """Get the request and client address from the socket. 
 
        May be overridden. 
 
        """ 
        return self.socket.accept() 
 
    def shutdown_request(self, request): 
        """Called to shutdown and close an individual request.""" 
        try: 
            #explicitly shutdown.  socket.close() merely releases 
            #the socket and waits for GC to perform the actual close. 
            request.shutdown(socket.SHUT_WR) 
        except socket.error: 
            pass #some platforms may raise ENOTCONN here 
        self.close_request(request) 
 
    def close_request(self, request): 
        """Called to clean up an individual request.""" 
        request.close() 
 
TCPServer
TCPServer
class ThreadingMixIn: 
    """Mix-in class to handle each request in a new thread.""" 
 
    # Decides how threads will act upon termination of the 
    # main process 
    daemon_threads = False 
 
    def process_request_thread(self, request, client_address): 
        """Same as in BaseServer but as a thread. 
 
        In addition, exception handling is done here. 
 
        """ 
        try: 
            self.finish_request(request, client_address) 
            self.shutdown_request(request) 
        except: 
            self.handle_error(request, client_address) 
            self.shutdown_request(request) 
 
    def process_request(self, request, client_address): 
        """Start a new thread to process the request.""" 
        t = threading.Thread(target = self.process_request_thread, 
                             args = (request, client_address)) 
        t.daemon = self.daemon_threads 
        t.start() 
 
ThreadingMixIn
ThreadingMixIn
class ThreadingTCPServer(ThreadingMixIn, TCPServer): pass
ThreadingTCPServer
class BaseRequestHandler: 
 
    """Base class for request handler classes. 
 
    This class is instantiated for each request to be handled.  The 
    constructor sets the instance variables request, client_address 
    and server, and then calls the handle() method.  To implement a 
    specific service, all you need to do is to derive a class which 
    defines a handle() method. 
 
    The handle() method can find the request as self.request, the 
    client address as self.client_address, and the server (in case it 
    needs access to per-server information) as self.server.  Since a 
    separate instance is created for each request, the handle() method 
    can define arbitrary other instance variariables. 
 
    """ 
 
    def __init__(self, request, client_address, server): 
        self.request = request 
        self.client_address = client_address 
        self.server = server 
        self.setup() 
        try: 
            self.handle() 
        finally: 
            self.finish() 
 
    def setup(self): 
        pass 
 
    def handle(self): 
        pass 
 
    def finish(self): 
        pass 
 
SocketServer.BaseRequestHandler
BaseRequestHandler

 SocketServer的ThreadingTCPServer之所以可以同时处理请求得益于 select 和 Threading 两个模块,其实本质上就是在服务器端为每一个客户端创建一个线程,当前线程用来处理对应客户端的请求,所以,可以支持同时多个个客户端连接。

ForkingTCPServer

ForkingTCPServer和ThreadingTCPServer的使用和执行流程基本一致,只不过在内部分别为请求者建立 “线程”  和 “进程”。

发布评论

分享到:

IT虾米网

微信公众号号:IT虾米 (左侧二维码扫一扫)欢迎添加!

isinstance和issubclass、动态模块导入、异常处理详解
你是第一个吃螃蟹的人
发表评论

◎欢迎参与讨论,请在这里发表您的看法、交流您的观点。