Showing posts with label programming. Show all posts
Showing posts with label programming. Show all posts

Tuesday, October 06, 2015

Mass kill processes in MySQL "show processlist"

So your server load is high you did "SHOW PROCESSLIST;" and you noticed many slow queries and you want to kill them.
If the list is too long to do it manually, you may want to try this way:

<?php

include("mySqlClass.inc.php");

$serv="localhost";
$user="root";
$pass="YOUR_PASS";
$database="SOME_DB";


$link = new mySqlClass();
$link->Connect($serv,$user,$pass,$database);
$result=$link->SqlQuery("SHOW FULL PROCESSLIST");
while ($result->NextRow()) {
    $process_id=$result->field["Id"];
    if ($result->field["Time"]>200) $link->SqlQuery("KILL $process_id");
}   

?>


and the mySqlClass.inc.php:


<?php

class mySqlClass {

  public $query_count;
  public $query_time;

  private $database;
  public $link;
  private $db_connected;

  function __construct()
  {
    $this->query_count = 0;
    $this->query_time = 0;
  }

  public function Connect($host, $user, $password, $database)
  {
    $this->database = $database;

    $this->link = @mysql_connect($host, $user, $password, true);

    if ($this->link)
    {
      if (@mysql_select_db($database, $this->link))
      {
        $this->db_connected = true;
        return true;
      }
    }

    $this->_error(mysql_errno(), mysql_error());
    return false;
  }

  public function SqlQuery($sql)
  {
    $time_start = explode(' ', microtime());

    if (!$this->db_connected) $this->_error(0, 'Error: MySQL DB Not Connected');

    $result_resource = @mysql_query($sql, $this->link);

    if (!$result_resource) $this->_error(@mysql_errno($this->link), @mysql_error($this->link));

    $obj = new SqlQueryResult($result_resource);

    if ($obj->RowCount() > 0)
    {
      // Return the first row of data results
      $result_array = @mysql_fetch_array($result_resource, MYSQL_ASSOC);

      if ($result_array)
      {
        while (list($key, $value) = each($result_array))
        {
          $obj->field[$key] = $value;
        }
      }
    }

    $time_end = explode (' ', microtime());
    $query_time = $time_end[1]+$time_end[0]-$time_start[1]-$time_start[0];
    $this->query_time += $time_end[1] + $time_end[0] - $time_start[1] - $time_start[0];
    $this->query_count++;

    return($obj);
  }

  public function QueryCount()
  {
    return $this->query_count;
  }

  public function QueryTime()
  {
    return $this->query_time;
  }

  private function _error($error_number, $error_text)
  {
    if ($error_number != 1141)
    {
      echo "Error #$error_number: $error_text";
      die();
    }
  }
}


class SqlQueryResult {

  public $field;

  private $result_resource;
  private $num_rows;
  private $current_row;

  function  __construct($result_resource)
  {
    $this->result_resource = $result_resource;
    $this->current_row = 0;
    $this->num_rows = @mysql_num_rows($this->result_resource);
    $this->field = array();
  }

  public function NextRow()
  {
    if ($this->current_row === 0)
    {
      // Row already fetched from SqlQuery() function
      // Do nothing
    }
    else if ($this->num_rows > 0 AND $result_array = @mysql_fetch_array($this->result_resource, MYSQL_ASSOC))
    {
      // This is the next iteration and there is a counted row returned
      // Grab data array
      if ($result_array)
      {
        while (list($key, $value) = each($result_array))
        {
          $this->field[$key] = $value;
        }
      }
    }
    else
    {
      // No more rows, end of data iteration
      // End result
      return false;
    }

    $this->current_row++;

    return true;
  }

  public function RowCount()
  {
    return $this->num_rows;
  }
}

Saturday, September 26, 2015

Allow PHP in Posts and preg_match

Allow PHP in Posts is a very cool plugin when you need to execute PHP in your posts, but you need a little time to get used to it, because it is kind of tricky. For instance I did not manage to use the preg_match function with square brackets. It simply replaces [a-z] with <a-z>. The only fix I could find is to use Code Snippets and add all the preg_match parts there, then use Shortcodes like [php function=5].

Also don't forget about [PHP debug=1] option which can be really useful.

Monday, September 21, 2015

remove accents and diacritics from a string in PHP

I needed to remove some diacritics from a text and the easiest way I've found is this:
$newstring = iconv('UTF-8', 'US-ASCII//TRANSLIT', $original_string);

Monday, April 19, 2010

DOS/Windows End of Line vs Linux EOLN

Working with bash scripts on some text files can be really annoying. Everything seems perfect but you just can't get the desired results.
One common reason for this is the incompatibility of the end of line markers between different operating systems. On Windows there are two characters: '\r\n' and on Linux only one: '\n'. And that extra '\r' can really mess your terminal and your echo outputs. On the other hand, taking a Linux file on a Windows notepad, will display everything on the same line. But don't worry, this can be fixed. If you are on Windows, instead of Notepad try Wordpad or Word and this will eventually display your file correctly.

On Linux, first you should check your file to see what you deal with. You can use hexdump or mc(midnight commander - mcedit).


$ hexdump dos_test.txt -C
[...] 6f 77 73 20 73 74 79 6c |DOS/Windows styl|
[...] 20 4c 69 6e 65 0d 0a |e End Of Line..|

$ hexdump lin_test.txt -C
[...] 75 78 20 73 74 79 6c 65 |Unix/Linux style|
[...] 20 45 4f 4c 4e 0a | EOLN.|


On the first file we have 0d 0a sequence and on the second line only 0a.

To avoid headaches when you use a Windows file with some bash scripts or something similar you need to convert it.

You can use tr:

$ tr -d '\r' inputfile.txt > outputfile.txt

or

$ dos2unix dosfile.txt unixfile.txt

You can also use AWK, PHP, SED to convert files, even ftp.

Wednesday, February 03, 2010

Include PHP files or code in phpBB HTML templates

Adding php code in your phpBB templates using the classical <?php ?> tags won't work.

To make things possible first go to your Administration Control Panel -> General -> Security Settings and enable Allow php in templates option.

Then you can use the following syntax to add php code:

<!-- PHP --> echo "PHP Code!"; <!-- ENDPHP -->

To include php files:

<!-- PHP --> include("/path/to/file.php"); <!-- ENDPHP -->


Saturday, October 06, 2007

Call to undefined function imagecreatefromjpeg()

I was trying to resize some jpeg file in a php script and got this error:

Fatal error: Call to undefined function imagecreatefromjpeg()

I have recompiled the PHP by adding the following parameters to the ./configure script: --with-gd --with-zlib --with-jpeg-dir and everything worked after.

Of course I had to fix some dependencies first, but it was easy under Debian using apt-get, I remember that I had to do something similar a few years ago under an old Red Hat and it was really painful to fix all those GD requirements.

Wednesday, August 15, 2007

Amazon AWS: php automated scripts using XML

I wanted to build some search engine friendly minisite with amazon XML feeds. Basically you choose a keyword and get results, eventually you get links to other keywords, all works recursively and so on. First of all, after I logged in into my amazon associates account I couldn't find anything good. I remember that when I was using mediaplazza feeds or peakclick feeds, everything was easy due to good and straight documentation. I couldn't find anything like this on amazon. I searched 3rd parties and found these which are pretty helpful: http://www.chipdir.nl/amazon/ and http://smallplanetonline.com/aws.php. They come with sources but anyway, instead of using those I preffered to write everything from scratch. I just look into how a query to http://xml.amazon.com/onca/xml3? should look like and then I played with the parameters and parsed everything using xml_parser function and with help of the examples from http://de.php.net/manual/en/function.xml-parser-create.php.
I just wish amazon could offer some good documentation and eventually some kind of simulator for the queries. But maybe they do but I was just too ignorant to see or they know where to hide it well. I got so pissed searching and I started to feel like I was waisting time and decided to do my own code.

Friday, February 02, 2007

JBoss: Hypersonic to mysql migration

So, now I had my server, deploying the trailbalzer

and even my application without any problems, but as there's already stated that Hypersonic isn't suited for production purposes and as I discovered that my tables were gone when restarting the server (only after some more hours I understoof it wasn't Hypersonic's fault, but a xml config file's), I decided to change to mySQL as a datasource for my applications using JBoss's services. Easy said...looking in a few
pages/articles/threads on this subject, the only thing I got to be sure about was
that there was no two opinions alike. And only for the 4.0.5 version, as I was careful not to read outdated info...Did those people offering advice really set their data source to mySQL? I guess I'll never know. Bottom line, none of the guides I'd found provided the complete solution to my problem, mainly all of them said the follwong things:

1.Download and set to the classpath the mysql driver (mysql-connector-java-5.0.4-bin.jar),
then copy this to [jboss-location]/server/default/lib directory.

2.Create a file named mysql-ds.xml in the
[jboss-location]/server/default/deploy/ directory and edit it similar
to
the example (my actual
file):


<?xml version="1.0" encoding="UTF-8"?>

<!-- $Id: mysql-ds.xml,v 1.3.2.3 2006/02/07 14:23:00 acoliver Exp $ -->
<!-- Datasource config for MySQL using 3.0.9 available from:
http://www.mysql.com/downloads/api-jdbc-stable.html

-->

<!-- This connection pool will be bound into JNDI with the name
"java:/MySqlDS" -->

<datasources>
<local-tx-datasource>
<jndi-name>MySqlDS</jndi-name>

<connection-url>jdbc:mysql://localhost:3306/OPA</connection-url>

<driver-class>com.mysql.jdbc.Driver</driver-class>
<user-name>A_USER_NAME</user-name>
<password>A_PASSWORD</password>
<min-pool-size>5</min-pool-size>
<max-pool-size>20</max-pool-size>
<idle-timeout-minutes>5</idle-timeout-minutes>



<exception-sorter-class-name>org.jboss.resource.adapter.jdbc.vendor.MySQLExceptionSorter</exception-sorter-class-name>
<!-- should only be used on drivers after 3.22.1 with "ping"
support

<valid-connection-checker-class-name>org.jboss.resource.adapter.jdbc.vendor.MySQLValidConnectionChecker</valid-connection-checker-class-name>
-->
<!-- sql to call when connection is created
<new-connection-sql>select 1</new-connection-sql>

-->
<!-- sql to call on an existing pooled connection when it is
obtained from pool - MySQLValidConnectionChecker is preferred for newer
drivers
<check-valid-connection-sql>select 1</check-valid-connection-sql>
-->

<!-- corresponding type-mapping in the standardjbosscmp-jdbc.xml
(optional) -->
<metadata>

<type-mapping>mySQL</type-mapping>
</metadata>
</local-tx-datasource>
</datasources>

The important lines are the bolded ones, one is for the url used for connecting to the mysql server; if you didn't change the port it listens to, leave 3306. 'localhost' is the host where the mysql server is running, if is something other than this, change it to the ip to reflect it corectly. After the slash there's the name of the datatbase the driver would connect to by default (in my case, "OPA").
The <user-name> element is obviously for specifying the user name that should be used when trying to connect to the mysql server, and the <password> element is for its corresponding password. Make sure u've created the user before testing the connection and have assigned that user with the rights for your above mentioned database. (see mysql manual for these operations :) )

3.Modify the file [jboss-location]\server\default\conf\standardjaws.xml
:


<?xml version="1.0" encoding="UTF-8"?>

<!DOCTYPE jbosscmp-jdbc PUBLIC
"-//JBoss//DTD JBOSSCMP-JDBC 3.0//EN"
"http://www.jboss.org/j2ee/dtd/jbosscmp-jdbc_3_0.dtd">

<!--
=====================================================================
-->
<!--
-->
<!-- Standard Jaws Configuration
-->

<!--
-->
<!--
=====================================================================
-->

<!-- $Id: standardjaws.xml 8624 2002-06-27 19:26:28Z dsundstrom $ -->

<jaws>
<datasource>java:/MySqlDS</datasource>
<type-mapping>mySQL</type-mapping>

<debug>false</debug>

<default-entity>
......{this part is unchanged}

4.Modify the file
[jboss-location]\server\default\conf\standardjbosscmp-jdbc.xml :

<?xml version="1.0" encoding="UTF-8"?>
<!DOCTYPE jbosscmp-jdbc PUBLIC
"-//JBoss//DTD JBOSSCMP-JDBC 4.0//EN"
"http://www.jboss.org/j2ee/dtd/jbosscmp-jdbc_4_0.dtd">

<!--
=====================================================================
-->
<!--
-->
<!-- Standard JBossCMP-JDBC Configuration
-->
<!--
-->
<!--
=====================================================================
-->

<!-- $Id: standardjbosscmp-jdbc.xml 41762 2006-03-06 14:39:33Z
aloubyansky $ -->

<jbosscmp-jdbc>

<defaults>
<datasource>java:/MySqlDS</datasource>
<!-- optional since 4.0
<datasource-mapping>mySQL</datasource-mapping> -->

<create-table>true</create-table>
<remove-table>false</remove-table>
........
Leave the long rest of this file unchanged. The <remove-table> is set to false so i would keep my application tables after a redeployment.
You don't have to change it from the default value of 'true' if you don't want this behaviour.

5.Add a <application-policy> element - inside the main <policy> one,
but not intercalated with other <application-policy> tags - in the
[jboss-location]\server\default\conf\login-config.xml file :


<policy>
.....
<application-policy name = "MySqlDbRealm">
<authentication>
<login-module code =
"org.jboss.resource.security.ConfiguredIdentityLoginModule"
flag = "required">
<module-option name =
"principal">the_mysql_username</module-option>
<module-option name =
"userName">the_mysql_username</module-option>

<module-option name
="password">the_mysql_pass</module-option>
<module-option name =
"managedConnectionFactoryName">jboss.jca:service=LocalTxCM,name=MySqlDS</module-option>
</login-module>
</authentication>
</application-policy>

....
</policy>

6.1. I don' really know if one could leave the jms need for a persistence environment to the old data source, HypersonicDS, and as I havent' used jms yet I can't be sure if I did the switching correctly (but by the lack of errors I'd say I did). But i proceeded to this step just to be sure:
Replace file [jboss-location]/server/default/deploy/jms/hsql-jdbc2-service.xml by
file [jboss-location]/docs/examples/jms/mysql-jdbc2-service.xml.

6.2.Rename hsqldb-jdbc-state-service.xml to mysql-jdbc-state-service.xml and change the line where the DeafultDS is mentioned:

<?xml version="1.0" encoding="UTF-8"?>

<!-- $Id: hsqldb-jdbc-state-service.xml 23386 2004-09-03 21:38:12Z
ejort $ -->

<server>

<!--
====================================================================
-->

<!-- JBossMQ State Management using HSQLDB
-->
<!-- See docs/examples/jms for other configurations
-->
<!--
====================================================================
-->

<!-- A Statemanager that stores state in the database -->
<mbean code="org.jboss.mq.sm.jdbc.JDBCStateManager"
name="jboss.mq:service=StateManager">
<depends
optional-attribute-name="ConnectionManager">jboss.jca:service=DataSourceBinding,name=MySqlDS</depends>

<attribute name="SqlProperties">
...

7. I can't stress out how important this was for me, it was the point missing in all other guides I read (many, so many of them....) :find all files containing a reference to DefaultDS and replace that reference with MySqlDS and do the same in replacing Hypersonic SQL with mySQL.

8. Don't forget to edit the persistence.xml in yout application to reflect the new data source used:

<persistence>
<persistence-unit name="the_name_of_your_persistence_unit">
<jta-data-source>java:/MySqlDS</jta-data-source>
<properties>

<property name="hibernate.hbm2ddl.auto"
value="update"/>
</properties>
</persistence-unit>
</persistence>

I must note that the "update" value in the line <property name="hibernate.hbm2ddl.auto" value="update"/> is also a part of the solution
for keeping the data in the tables after a redeploy of the application, instead of losing all the previous populated tables.

problems installing JBoss

I had some problems installing JBoss 4.0.5 with jems, although that's
the way they recommend it on the official site. The problem appeared at
JBoss server startup and was mainly consisting in these errors
appearing in the console:

16:23:42,873 INFO [TomcatDeployer] deploy, ctxPath=/web-console, warUrl=.../deploy/management/conso
le-mgr.sar/web-console.war/
16:23:43,634 INFO [[/web-console]] MonitorsFolder: Failed to init
plugin, null
16:23:43,644 INFO [[/web-console]] UCLs: Failed to init plugin, null
16:23:43,664 INFO [[/web-console]] JMS Message: Failed to init plugin,
null
16:23:43,664 INFO [[/web-console]] JSR77 WebModule: Failed to init
plugin, null
16:23:43,724 INFO [[/web-console]] J2EEFolder: Failed to init plugin,
null
16:23:43,754 INFO [[/web-console]] AOPFolder: Failed to init plugin,
null
16:23:43,754 INFO [[/web-console]] SystemFolder: Failed to init
plugin, null
16:23:43,804 INFO [[/web-console]] MBeans: Failed to init plugin, null
16:23:43,814 INFO [[/web-console]] JSR77 Domains and Servers: Failed
to init plugin, null
16:23:43,814 INFO [[/web-console]] JSR77 EJBModules and EJBs: Failed
to init plugin, null
16:23:43,824 INFO [[/web-console]] JSR77 J2EE Apps: Failed to init
plugin, null
16:23:47,870 INFO [MailService] Mail Service bound to java:/Mail
...


After some digging, I found out that there was a bug in the jems
installer 1.2.0 CR1 that led to a line missing in a config xml file
(\jboss-4.0.5.GA\server\default\deploy\management\console-mgr.sar\web-console.war\WEB-INF\jboss-web.xml).
The obvious solution was, as a JBoss poster stated on the forum, to put
the missing line in place (see this ),
but disappointingly this didn't work for me. I really needed a EJB 3.0
supporting version of JBoss 4.0.5, and after some more hassle with the
jems installer, I finally decided to abandon this path and go directly
to the source code. I unpacked it and built it with ant (the build.xml
is provided) - for some odd meaningless reason the build.sh failed to
run - then got my ejb 3.0 enabled server in
jboss-4.0.5.GA-src\build\output\jboss-4.0.5.GA-ejb3 folder. Note that jdk1.5 is required for this,
no earlier version would do for ejb 3.0. That's about it, I hope I
didn't forget impotant details, hopefully this would spare some people the
pain of going through the same countless tries of installing a good
version of JBoss 4.0.5 server.

I had some troubles with migrating from the default JBoss data
source (Hypersonic) to mysql, but I'll write about this later.

Thursday, February 01, 2007

phplinkdirectory errors

Today I got this strange error:

Warning: Smarty error: validate: validator id 'v_user' is not registered. in /home/dir/libs/smarty/Smarty.class.php on line 1088


After some investigations I noticed my server's disk was out of space.
So removing some big logs which were taking most of the space, fixed the problem.

Tuesday, June 27, 2006

HeaderName and PHP scripts execution problem

If you are using mod_autoindex to list your directory contents on the web, you may want to add header and footer using HeaderName and
ReadmeName directives.

However, if you refer to .php scripts in these directives, they are
either ignored or listed as plain text(php source).

In order to have HEADER.php really executed, you need to add in your httpd.conf some code like:
 
<Directory /dir>
AddType text/html .php
HeaderName /HEADER.php
<Files "*.php">
AddHandler application/x-httpd-php .php
</Files>
</Directory>

Thursday, June 08, 2006

PHPMyAdmin freezes on databases with too many tables

If you are using phpmyadmin on a database with more 1500 or more tables, you may experience problems. The page will not load at all so you will not be able to operate propery with phpmyadmin.

Of course you can still run commands from a terminal or console but it is not as confortable as using phpmyadmin.

After digging a lot in the config.default.php filesand trying diferent results and limits for various variables, I found out that problems comes from PHP itsels. So the first thing you should do, is to raise the memory_limit in php.ini . It worked for me.